Web question answering presentation

139 views

Published on

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
139
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Web question answering presentation

  1. 1. Is More Always Better ? Dumais et al, Microsoft Research A presentation by George Karlos
  2. 2.  Combines information retrieval and natural language processing  Answer questions (posed in natural language)  Retrieve answers  No documents  No passages What is a QuestionAnswering System ?
  3. 3.  TREC Question AnsweringTrack ▪ Fact-based, short-answer questions  “Who killed Abraham Lincoln?”  “How tall is Mount Everest?”  Dumais et al’s System ▪ Focus on same type of questions ▪ Techniques are more broadly applicable
  4. 4. More Data Higher Accuracy Use Web as source Other QA groups:  Variety of linguistic resources (part-of–speech tagging, syntactic parsing, semantic relations, named entity extraction, dictionaries, Wordnet, etc . ) Dumais et al’s System:  WEB Redundancy !!! multiple, differently phrased, answer occurrences
  5. 5. Small information source  Hard to find answers  Likely only 1 answer exists  Complex relations between Q & A ▪ Anaphor resolution, synonymy, alternate syntactic formulations, indirect answers (NLP) Web  More likely to find answers in simple relation to the question. Less likely to deal with NLP systems difficulties.
  6. 6.  Enables Simple Query Rewrites  More sources that include answers in a simple, related to the question form. e.g. “Who killed Abraham Lincoln?” 1. ______ killed Abraham Lincoln. 2. Abraham Lincoln was killed by ______ etc.
  7. 7.  Facilitates Answer Mining  Improves efficiency even if no obvious answers are found. e.g. “How many times did Bjorn Borg winWimbledon?” 1. Bjorn Borg <text> <text> Wimbledon <text> <text> 5 <text> 2. Wimbledon <text> <text> <text> Bjorn Borg <text> 37 <text> 3. <text> Bjorn Borg <text> <text> 5 <text> <text> Wimbledon 4. 5 <text> <text> Wimbleton <text> <text> Bjorn Borg 5 is the most frequent number  Most likely the correct answer.
  8. 8. Four main components:  Rewrite Query  N-gram Mining  N-gram Filtering/Reweighting  N-gramTiling
  9. 9. Rewrite query in a way that a possible answer might be formed e.g. “When was Abraham Lincoln born?” “Abraham Lincoln was born on <DATE>”  7 categories of questions  Different sets of rewrite rules  1-5 rewrite types Rewrite Query
  10. 10. Output : [string , L/R/- , weight] Rewrite Query Reformulated search query Position answer is expected How much the answer is preferred “Abraham Lincoln was born on”  more likely to be correct “Abraham” AND “Lincoln” AND “Born” High precision query  Lower precision query 
  11. 11. “The Louvre Museum is located in” 1. W1 ISW2 W3 ….Wn 2. W1W2 IS W3….Wn … etc.  More rewrites  Proper rewrite guaranteed to be found  Using a parser can result in proper rewrite not be found Rewrite Query Simple string manipulations “Where is the Louvre museum located?”  Final rewrite: ANDing of non-stop words “Louvre Museum” AND “located” StopWords ( in, the, etc..) • Important indicators of likely answers
  12. 12. N-Grams  Collections of words or letters that frequently appear on the web 1. 1-,2-,3-grams are extracted from the summaries 2. Scored based on the weight of the query rewrite that retrieved them 3. Scores summed 4. Final score based on rewrite rules and number of unique summaries in which it occurred N-Gram Mining
  13. 13.  Query assigned one of seven question types (who, what, how many, etc..)  System determines what filters to apply ▪ Boost score of a potential answer ▪ Remove strings from the candidate list  Answers analyzed and rescored N-Gram Filtering/Reweighting
  14. 14.  Answer tiling algorithm ▪ Merges similar answers ▪ Assembles longer answers out of answer fragments e.g. “A B C” and “B C D”  “ A B C D ” Algorithm stops when no n-grams can be further tiled. N-GramTiling
  15. 15. AND Rewrites Only,Top 100 (Google) MRR NumCorrect PropCorrect Web1 0.450 281 0.562 TREC, Contiguous Snippet 0.186 117 0.234 TREC, Non-Contiguous Snippet 0.187 128 0.256 •TREC vs.WEB Lack of redundancy inTREC ! AND Rewrites Only,Top 100 (MSNSearch) MRR NumCorrect PropCorrect Web1 0.450 281 0.562 TREC, Contiguous Snippet 0.186 117 0.234 TREC, Non-Contiguous Snippet 0.187 128 0.256 Web2, Contiguous Snippet 0.355 227 0.454 Web2, Non-Contiguous Snippet 0.383 243 0.486
  16. 16. •TREC &WEB(Google) Combined •Larger source. Better? Combined QA results Trec 0.262 MRR Web 0.416 MRR MRR: 0.433 NumCorrect: 283  4.1% improvement
  17. 17.  QA Accuracy and number of snippets  Smaller collections  Lower Accuracy  Snippet quality less important More is better up to a limit

×