Understanding User Goals in Web Search Daniel E. Rose Danny Levinson
Not only what, but also why● previous work on information-seeking behavior● authors hierarchy of search goals● how to classify queries● analysis of results● applicability of model
LIS precedents ● Bates ○ "Information Search Tactics" (1979) ○ "The Design of Browsing and Berrypicking Techniques for the Online Search Interface " (1989) Above: Bates berrypicking model Left: the traditional model of document recovery Diagrams are recreations of figures that appear in "The Design of Browsing and Berrypicking Techniques for the Online Search Interface."
● Belkin, Oddy, and Brooks ○ "ASK for Information Retrieval" (1982) ■ from "Part I. Background and Theory"
■ from "Part II. Results of a Design Study"
● Broder ○ "A Taxonomy of Web Search" (2002)
Process● developed flat list of user goals● used list to classify queries in test set● revised categories as necessary--developing hierarchical classification● manually classified three sets of queries ○ Each set of approximately 500 queries was randomly selected from AltaVista query logs on a different day, and at a different time of the day, than the other two.
Search goal hierarchy
Search goal hierarchy (continued)
Manual query classificationObjects of consideration: "We need to know the relative ● the query itself prevalence of various goals. And ● results returned by the if we hope to infer goals search engine automatically in the future, we ● results clicked on by the need to know that it is possible user to do so manually" (16). ● further searches or other actions by the userDo these documents provide "Once we could successfully"sufficient information for a classify queries manually, wehuman to consistently classify would be able to provide trainingqueries according to our goal data for a future automaticframework" (16)? classification system" (16).
Final Fantasy exampleTime Delta t Event Details What kind of a site did the user who searched for "final36 36 result click pg 1, pos 1 http://www. ffonline.com fantasy" intend to find--a site that sells a version of the113 77 query pg 1 final fantasy game, one that lists the118 5 result click pg 1, pos 8 http://www. games "official" rules, or one eyesonff.com that provides less exact147 29 result click pg 1, pos 8 http://www. information about the game? eyesonff.com Examining the results returned by two search engines for this query, and the users subsequent clicks, the authors conclude that the goal of this search was "undirected" information.
Applications and limitations● "If our findings about ● "One issue is that we the relatively small have no way of knowing number of navigational conclusively whether the queries are accurate, goal we inferred for a they suggest that much query is in fact the user’s of the attention in the actual goal. In the future, commercial search we would like to combine engine world may be our work with user misdirected" (18). studies, including qualitative data such as diary reports of user goals" (19).
Questions ● How does Rose and Levinsons user goals paradigm compare to the idea of document relevance? Does a result need to meet a search goal in order for the user to perceive it as relevant, or simply convey information related to the search query? ● In what ways does their user goals model take into account query refinement and/or query expansion? ● What is the users goal in the Aloha/American Airlines scenario (outlined on slides 8 and 9)? ● How would you apply Rose and Levinsons research to the following questions: ○ Why does a search occur? ○ What is its purpose? ○ How does it proceed?