Query recommendation papers


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • All questions are associated with an implicit answer type. So even if we don't know the actual answer, we can expect what the type of that answer would be. An accurate expectation makes it easier to predict the answer from a sentence that contains the query words. <example>
  • The answer to a question like “What are the tourist attractions in Reims?” could be many different things – church, historic building,, park, statue, famous intersection etc.
  • Unsupervised method to dynamically construct a probabilistic answer type model for each question. Such a model evaluates whether or not a word fits into the question context. <example> we can find words that appear in this context from a corpus.
  • Parsed the AQUAINT corpus (3GB) with Minipar and collected the frequency counts of words appearing in various contexts. Parsing and database construction is done offline as the database is identical for all questions. Extracted 527, 768 contexts that appeared at least 25 times in the corpus. Which city hosted the winter olympics? Question clearly states that the desired answer type is city. So the context is “X is a city”
  • The first model assigns the same likelihood to every instance of the candidate word. Since a word can be polysemous like we saw in the “washington” example, we introduce the candidate context. Various parameters of the model are then estimated using the context filler database and appropriate probability distribution
  • The model is used to filter the contents of the documents retrieved by IR portion of the question-answer system. Each answer candidate is scored and the list is sorted in descending order of score. Then treat the system as filter and observe the number of candidates that must pass through it before at least one correct answer is accepted. A model that allows low percentage of candidates to pass while still accepting at least one answer is favourable to one that passes many candidates. Compared against two models – Oracle system that uses manual question classification and manual entity tagging. And ANNIE that performs automatic tagging.
  • Users submit a query to search according to their potential information need. They then consecutively reformulate their queries in a search session until their original needs are fulfilled.
  • Idea is to integrate the learned search intents of queries into the prior preference of the personalized random walk, and apply the random walk under different search intent resp. Lambda is the teleportation probability rho is the weight balancing the original query and its intent. It is set less than 1 to smoothen the preference vector with the learned intents which can provide rich information about the original query.
  • Query recommendation papers

    1. 1. Summary of Query Recommendation papers Ashish Kulkarni
    2. 2. A Probabilistic Answer Type Model
    3. 3. Introduction <ul><li>What is the capital of Norway? </li></ul><ul><li>We would expect the answer to be a city and can filter out most text from </li></ul><ul><ul><li>The landed aristocracy was virtually crushed by Hakon V, who reigned from 1299 to 1319 and Oslo became the capital of Norway, replacing Bergen as the principal city of the kingdom. </li></ul></ul><ul><li>The goal of answer typing is to determine whether a word's semantic type is appropriate as an answer for a question </li></ul>
    4. 4. Previous approaches to Answer Typing <ul><li>Used a predefined set of answer types and used supervised learning or manually constructed rules </li></ul><ul><li>There will always be questions whose answers do not belong to any of the predefined types. “What are tourist attractions in Reims?” The answer could be many different things. Define a catch-all class. Not as effective as the other answer types. </li></ul><ul><li>Granularity – if the types are too specific they are difficult to tag. If they are too general, too many candidates might be identified. </li></ul>
    5. 5. Proposed approach <ul><li>Unsupervised method to dynamically construct a probabilistic answer type model </li></ul><ul><li>“ What are the tourist attractions in Reims?” We would expect the answers to fit into the context “X is a tourist attraction.” From a corpus we can find words that appeared in this context. </li></ul><ul><li>Using the frequency counts of these words in the context, we construct a probabilistic model to compute the probability for a word w to occur in a set of contexts T given an occurrence of w – P(in(w, T) | w). </li></ul><ul><li>Parameters of this model are obtained from an automatically parsed, unlabelled corpus. By asking whether a word would occur in a particular context extracted from a question, we avoid explicitly specifying a list of possible answer types. </li></ul>
    6. 6. Resources used by the model <ul><li>Word clusters – abstracting a given word to a class of related words. Clustering by Committee (CBC) algorithm. A word may belong to multiple clusters. </li></ul><ul><li>Contexts – context in which a word appears imposes constraints on the semantic type of the word. Represented by undirected paths in the dependency trees involving the word at the beginning or end. The word itself is replaced by X. A word is said to be filler of the context if it replaces X. </li></ul><ul><li>Question contexts are extracted from a question. An answer is a plausible filler of a question context. Two rules – if wh-word has a trace in the parse tree, the question contexts are the contexts of the trace. If wh-word is a determiner, then single context involving the noun that is modified by the determiner. </li></ul><ul><li>Candidate contexts are extracted from parse trees of the candidate answers. Occurrence of washington in 'Washington's descendants' and 'suburban Washington' should be scored differently if the question is seeking a location. </li></ul>
    7. 7. Probabilistic model <ul><li>Goal is to evaluate the appropriateness of an answer candidate. This is proportional to the probability that it will be a filler of the question context Tq extracted from the question. P(in(w, Tq)|w). </li></ul><ul><li>To mitigate data sparseness, variable C for clusters is introduced. It can be shown that the above model splits into two parts – one that models which clusters a word belongs to and the other that models the appropriateness of the cluster to question contexts. </li></ul><ul><li>Then introduce the candidate context and compute joint likelihood P(in(w,Tq) | w, in(w, Tw)), where Tw is the set of contexts for the candidate w. </li></ul>
    8. 8. Conclusions <ul><li>Explicit answer types can exhibit poor performance especially for those questions not fitting one of the types. </li></ul><ul><li>The answer types also need to be redefined when the domain or corpus changes significantly. The probabilistic answer typing model can adapt to different corpora and question answering domains. </li></ul><ul><li>This model can be combined with other existing answer typing strategies especially in those cases where catch-all classes are used. </li></ul>
    9. 9. Exploring the Query-Flow Graph with a Mixture Model for Query Recommendation
    10. 10. Introduction <ul><li>Query – flow graph: Nodes represent unique queries and two nodes are connected by a directed edge if they occur consecutively in a search session. A weighing function assigns weight representing the probability that two nodes q and q' are part of the same chain. </li></ul><ul><li>Chain is defined as a sequence of queries with similar information need. </li></ul>Query-flow graph Results, sessions, clickthrough Query Logs Query Recommendations
    11. 11. Previous approach to query recommendation <ul><li>Traditionally, personalized random walk over query-flow graph was used for recommendation. </li></ul><ul><li>Dangling queries </li></ul><ul><ul><li>No out links </li></ul></ul><ul><ul><li>Nearly 9% of whole queries </li></ul></ul><ul><li>Ambiguous queries </li></ul><ul><ul><li>Mixed recommendation </li></ul></ul><ul><ul><ul><li>Hard to read </li></ul></ul></ul><ul><ul><li>Dominant recommendation </li></ul></ul><ul><ul><ul><li>Cannot satisfy different needs </li></ul></ul></ul>
    12. 12. Proposed approach <ul><li>Novel mixture model to interpret the generation of the query-flow graph under multiple hidden intents. </li></ul><ul><li>Assumptions </li></ul><ul><ul><li>Queries are generated from some hidden search intents </li></ul></ul><ul><ul><li>Two queries occurred consecutively in a session if they are from the same search intent. </li></ul></ul><ul><li>Two step process </li></ul><ul><ul><li>Apply a novel mixture model over query-flow graph to learn the intents of queries </li></ul></ul><ul><ul><li>Perform an intent-biased random walk on the query-flow graph for recommendations. </li></ul></ul>
    13. 13. Mixture model <ul><li>The Mixture model is a probabilistic model of generating query-flow graph </li></ul><ul><li>The approach uses machine learning techniques to train the model to learn intents of the search queries. </li></ul>
    14. 14. Intent-biased random walk <ul><ul><li>The model adds the query intent to the preference vector </li></ul></ul><ul><ul><li>For dangling queries, the preference vector backs off to query’s intents to recommend the related queries. </li></ul></ul><ul><ul><li>For ambiguous queries, run the intent-biased random walk for every intents of the query and group the recommendations by intent. </li></ul></ul>
    15. 15. Experiment results <ul><li>Data set </li></ul><ul><ul><li>3-month query log from a commercial search engine </li></ul></ul><ul><ul><li>Query stream split into sessions using 30 min timeout </li></ul></ul><ul><ul><li>The biggest connected graph is extracted for experiments, which consisted of 16,980 queries and 51,214 edges. </li></ul></ul><ul><li>Mixture model with 600 dimensions (hidden intents) </li></ul>Dangling query suggestions Ambiguous query suggestions
    16. 16. Toward a deeper understanding of user intent and query expressiveness
    17. 17. Introduction <ul><li>Inferring users' intents is a popular research topic </li></ul><ul><li>Intent is the “need behind a query” </li></ul><ul><li>What is in the user's mind when they enter a search query? </li></ul>
    18. 18. Previous approach to identifying intent <ul><li>Focussed on intent modeling or on clustering intents into categories </li></ul><ul><li>They assume that human annotators can infer the main intent of each query </li></ul><ul><li>They suffer from the human judgement of inferring the intent by observing the queries and/or the URLs browsed </li></ul><ul><li>Need to understand - </li></ul><ul><ul><li>What constitutes an intent? </li></ul></ul><ul><ul><li>What factors go into articulating what is in user's mind into queries? </li></ul></ul>
    19. 19. Proposed approach <ul><li>Factors to evaluate the effectiveness of a set of queries to articulate the user intent </li></ul><ul><ul><li>Complexity of the task user wants to accomplish </li></ul></ul><ul><ul><li>Number of dimensions already explored </li></ul></ul><ul><ul><li>Specificity of what user is looking for </li></ul></ul>
    20. 20. Empirical study <ul><li>10 participants performed number of search tasks </li></ul><ul><li>Listed 10 things they were interested to search (before session) </li></ul><ul><li>55 min search session </li></ul><ul><li>Not required to complete each search topic in their list </li></ul><ul><li>Quantitative data collected through video recording </li></ul><ul><ul><li>Queries </li></ul></ul><ul><ul><li>Search results </li></ul></ul><ul><ul><li>Clicked URLs </li></ul></ul><ul><ul><li>Landing pages </li></ul></ul><ul><ul><li>Dwell time etc. </li></ul></ul>
    21. 21. Sample search tasks <ul><li>Peruvian literature: intent was to replace some readings from a book with easier and shorter readings from the web. User followed the structure of the book (chapters, topics) while querying. </li></ul><ul><li>Bugaboo stroller: intent is to sell an used stroller at the right price. In the queries user did not convey the intent to sell vs buy. </li></ul><ul><li>San Francisco restaurants: looking for a restaurant user hasn't been to before. Wants to explore what is out there and also find something fun to do after dinner. </li></ul><ul><li>A's spring training: intent is to find the right resource for organizing an A's spring training. </li></ul><ul><li>Things to do in Vegas: intent is to look for things that others have done that the user has not done already. </li></ul>
    22. 22. Analysis <ul><li>Analysis based on answering following questions - </li></ul><ul><ul><li>What is the complexity of the search task? </li></ul></ul><ul><ul><li>What is the known component of the intent? </li></ul></ul><ul><ul><li>What is the unknown component of the intent? </li></ul></ul><ul><ul><li>How much effort does the searcher have to put to complete the search task? </li></ul></ul><ul><ul><li>Is the intent articulated well by the set of queries? </li></ul></ul>
    23. 23. Observations <ul><li>Intents have a structure that reflects the user's mental model at the time of their search </li></ul><ul><li>There can be conjunction of intents </li></ul><ul><li>Intent could also be what someone does not want/need </li></ul><ul><li>Not all queries convey the intent </li></ul>
    24. 24. Discussions <ul><li>Depending on whether the intent is general or specific, the effectiveness of the queries can vary. It is not enough to consider a single query in the session to predict the intent. </li></ul><ul><li>Personalization – result set modification based on the user's current knowledge </li></ul><ul><li>Diversification of search results is inadequate to compensate for the search engine's lack of knowledge of what is in user's mind </li></ul>
    25. 25. Thank You