Arabic question answering ‫‬

498 views

Published on

Published in: Technology, Education
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
498
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
26
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Arabic question answering ‫‬

  1. 1. Imam UniversityCollege of Computer and Information systemsComputer sciences DepartmentArabic Question Answering :by Asma Ahmad Asma alharbinadia AL-MutiriSupervised by: Dr .Amal Al seefSecond semester :1434-14352013
  2. 2. Arabic Question AnsweringOverview:O The implementation of Arabic Question-Answering system components .O QASAL & QARAB System components.O Yes/No Arabic Question Answering.
  3. 3. ARABIQA GENERICARCHITECTURE
  4. 4. Named Entity RecognizerO A NER system identifies propernames, temporal and numeric expressions .O in this Arabic NER system is based MEapproach.O For the proper names recognition:O For temporal and numeric expressions: istotally based on patterns and a smalldictionary containing the names of days andmonths in Arabic, and numbers written inletters.
  5. 5. The implementation of ArabicQuestion-Answering systemO NooJ is a linguistic environment thatincludes large-coverage dictionaries andgrammars.O a spell-checker that corrects the mostfrequent errors.O a named entity recognition tool which isset of rules described into local grammars
  6. 6. QASAL Systemcomponents
  7. 7. Question analysis: this step it is apply the set oflinguistic resources to the input question.For example shows the NooJ’s text annotationstructure that gives the linguistic analysis of eachword form in our sample question
  8. 8. Passage retrieval: The first task of this stepcould be the selection of one or moreautomatically extract the answer of theinput question.
  9. 9. Answer Extraction: this last step uses thedisplayed concordance table toautomatically extract the answer of theinput question.Example1 :Answer Extraction for the factoid question:
  10. 10. Example 2:
  11. 11. QARAB SystemfNLB ToolQuestionQuestionanalyzerIR RankedDocumentsPassageselectionHypothesizedAnswerAl-RayaNewspaperDocumentfull IRsystem
  12. 12. Information Retrieval system .O To search the document collection to selectdocuments containing information relevant to theuser’s query.O Lundquist et al. [1999] IR system that can beconstructed using a relational database managementsystem (RDBMS).O But in this paper it contain following databaserelations:1. ROOT_TABLE.2. STEM_TABLE.3. POSTING_TABLE.4. DOCUMENT_TABLE.5. PARAGRAPH_TABLE.
  13. 13. The NLb systemThe NLB model is:1. Tokenizer.2. type finder.3. feature finder.4. proper noun phrase parser.
  14. 14. How to extract the AnswerAssume the user posed the following question toQARAB:The IR return this passage . How?!
  15. 15. Step1:O performing token and remove the stopword of question , Then tagging the wordfor POS.
  16. 16. Step 2:O QARAB constructs the query as a “bag ofwords” and passes it to the IR system.
  17. 17. ExampleStep 3: Determine the expected type of the answer:Who? >>> personal name.Step4: Generating the answer.
  18. 18. Yes/No ArabicQuestion Answering
  19. 19. SYSTEM ARCHITECTURE:QuestionAnalysismoduleTextretrievalmoduleAnswerSelectionmodule
  20. 20. Question AnalysisO Removing the question mark.O Removing the interrogative particleO Tokenizing: the tokenizer divides the userquestion into its separate words .Andnormalize the (Alef) letter.O Removing the stop words.O Removing the negation particles. (if itexits) and set the negation property of thequestion representation
  21. 21. Question AnalysisO Tagging: to determine the type of aword, verb or noun and obtain its root.O Parsing: recall that the Arabic sentenceafter the interrogative particle is nominalor verbal.
  22. 22. Question AnalysisIn nominal sentence, we are interested with thebeginning noun “topic” ( ) which is the firstnoun after the interrogative particle ( ). And thecomment noun ( ) and we can mark it as thelast noun without the article ( ).In verbal sentence we are interested with theverb of the sentence which occur immediatelyafterthe interrogative particle ( ) , and the subjectthat follow the verb.
  23. 23. Question AnalysisLogical Representation(With Nominal Sentences)Affirmative questionsO N (Topic, root (Comment), root({remaining words }))O N (Topic, root (Comment Synonyms), root({remaining words}))O ~N (Topic, root (Comment Antonyms), root({remaining words}))
  24. 24. Question AnalysisLogical Representation(With Nominal Sentences)O Negated questions :O ~N (Topic, root (Comment), root({remaining words}))O ~N (topic, root (Comment Synonyms), root({remaining words}))O N (Topic, root (Comment Antonyms), root({remaining words}))
  25. 25. Question AnalysisO Example-----<synonymO N( , root ( ),root( ))O N( , root ( ),root( ))
  26. 26. Question AnalysisLogical Representation(With Verbal Sentences)Affirmative questions :O V (Subject noun, root (verb), root ({remaining words}))O V (Subject noun, root (verb Synonyms), root ({remainingwords}))O ~V (Subject noun, root (verb Antonyms), root({remaining words}))
  27. 27. Question AnalysisLogical Representation(With VerbalSentences)Negated questionsO ~V (Subject noun, root (verb), root({remaining words}))O ~ V (Subject noun, root (verbSynonyms), root ({remaining words}))O V (Subject noun, root (verbAntonyms), root ({remaining words}))
  28. 28. Question AnalysisExample---<AntonymO V( , root ( ),root( ))O ~V( , root ( ),root( ))
  29. 29. Text Processing & RetrievalThey are 20 documents in corpus. This module uses twotechniques to retrieve the top 5candidate paragraphs (with variable length (that are mostrelevant to the user question:O Paragraphs technique: - Split the documents into itsbuilt-in paragraphs and retrieve the top 5 paragraphsregardless from which document they are, according tosome indexing scheme.O Document technique-:Retrieve the top 5 documentsafter they are ranked, then use the first indexing schemeto retrieve the top 5 paragraphs.
  30. 30. Answer Selection &generationAfter the 5 paragraphs are selected usingdocuments technique or paragraphstechnique, we need to select the bestsentence to represent the answer, andaccordingly generates yes or no .
  31. 31. Answer Selection &generationO Split the paragraphs into their sentences .O In normal sentences we are interested inthe exact topic ( ) not its used root, sowe omit each sentence that does notcontain it (in the original form )In verbalsentence we are interested in the exactsubject ( ) not its used root , so we omiteach sentence that does not contain it (inthe original form )
  32. 32. Answer Selection &generationO In the result sentence , we look for theremaining terms (in root form) that derivedfrom thequestion in the logical representation (exceptthe subject or the topic ), if the they exist, assignthose indexes according to their position in thesentence. So each sentence will have its ownrankas follow :Rank =last occurrence - first occurrenceO look for ( ) negation particles in theselected answer (if exist).
  33. 33. Answer Selection &generationO Using the selected answer and the logicalrepresentation of the question to generateyes ,or no a follows :1. Yes ,if : The question and the answerare affirmative .The question and theanswer are negated.2. No, if :The question if affirmative and theanswer are negated.The question isnegated and the answer is affirmative.
  34. 34. EXPERIMENTS ANDRESULTS69%Arabic QA system97.3%Arabic Q-A usesQARAB83.3%PR system
  35. 35. conclusionO We have described the genericarchitecture for AQ answerO compare with deferent systemO How presses the question and give theanswers.

×