Addaall Arabic Search Engine: Improving Search based on Combination of      Morphological Analysis and Generation Consider...
emphasized.                                                                    root in Arabic is the part which contains t...
following:    And finally there is the Root Level, representing the    most possible search in the context of semantic    ...
Aljlayl M. (2002). Arabic Search: Improving the Retrieval    Effectiveness via a Light Stemming Approach. ACM    Eleventh ...
Upcoming SlideShare
Loading in …5

Addlaall search-engine--hattab-haddad-yaseen-uop


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Addlaall search-engine--hattab-haddad-yaseen-uop

  1. 1. Addaall Arabic Search Engine: Improving Search based on Combination of Morphological Analysis and Generation Considering Semantic Patterns Mamoun Hattab*, Bassam Haddad†, Mustafa Yaseen‡, Asem Duraidi* and A. Abu Shmais* *Arabic Textware Inc., Amman-Jordan {m.hattab, a.duraidi, a.abushmais} † University of Petra, Department of Computer Science, Amman-Jordan ‡ Amman University, Department of Computer Science, Amman-Jordan AbstractThis paper addresses some issues involved in utilization of different levels of Arabic morphological knowledge in improving thesearch process in a search engine. The different levels of morphological knowledge can be considered as an incremental processconsidering the next sensitive word pattern in the context of enhancing the possibility of covering a higher recall and precision. 1. IntroductionThis paper attempts to demonstrate how the utilization of need for a linguistic approach to handle the search processdifferent levels of Arabic morphological knowledge can (Brin S. and Page L., 1998).improve the searching process in a Search Engine. We can Addaall1 Arabic Search Engine built by Arabic Textwareshow how combining morphological analysis and Inc2. utilizes a morphological analyzer and generator togeneration can be used for retrieving information based construct different indices based on both the root and stemon semi-semantic linguistic search in Arabic. This of a word. While retrieving information based on the roottechnique extends the usage of roots and stems into a overflows the user with a complete but less relevant set ofmethod of categorization for morphological patterns into results, the use of the stem based search not only retrievessemantic groups, and uses a morphological generator to results in lesser numbers but also the set of results is lessproduce the different inflections of a word to be retrieved accurate.based on the semantic distance from the word.Google approach is primary based on the matchingprocess between the user’s key words and the texts under 2. Morphological Knowledge implieswhich it is indexed in terms of words (Brin S. and Page Syntactic and Semantic IndicatorsL., 1998). Words are treated as a set of symbols, notwords with meanings to human users. And while plain There are much information produced by thelinguistic measures, such as stemming and structured data morphological analysis which are useful in the next stepssearch are used to enhance results, in nature Google and of processing such as the syntactic and the semanticsuch search engines are still "Symbolic Computing" analysis. Some of the information is:machines.In third generation of search engines, Natural Language The morphological type of a wordProcessing technologies are applied in searching We mean by "morphological type" determining if theextensively, because in the first place, the search is seen word is noun, verb, or particle; and what subtype ofas a language understanding process. This approach for nouns, verbs, particles it is. It is clearly noticed thatsearch is paradigmatically different and a level higher in the syntactic functionality of the word is highlyterms of the degrees of system difficulty and complexity. dependent on the morphological type of the word. ForAnd it offers more accurate and consistent search results example, the syntactic function of a "subject" requiresthan the second generation search engines through its that the morphological type is a noun. Theintelligence in language understanding. In this context morphological type provides the semantic analysismany researchers have considered this aspect (Abuleil and with the semantic properties of each word. ForAlsamra, 2004; Chen and Gey, 2002; Aljlayl M., 2002). example, the determining of (/‫ ,/ﻓَـﻌﹼﺎﻝ‬fa،، āl) to be anFor a language like Arabic where the structure of a wordcan change according to many factors while maintaining exaggeration form tells that the meaning is muchthe same meaning, "Symbolic Computing" does notretrieve an accurate set of results and there is an immense 1 2 159
  2. 2. emphasized. root in Arabic is the part which contains the meaning; it is the core of language. Anyway, some roots has The Determiner We mean by "determiner" any indeclinable noun, verb, syntactic properties like (/‫ )/ ﺣﺴﺐ‬which tells that the determiner or particle in addition to the affixes. It is verb needs two objects. clear that the variety of the syntactic functions which the determiners can do is wide. The proposition (/‫,/ ﰲ‬ 2.1 Morphological Levels for Search fī, in) provides that the word follows is a noun and genitive (Haddad B., 2007). Even the letters such as To measure the closeness between the meaning of the word being searched for, and the suggested words to be (/‫,/ ﻭ‬ wāw) in (/‫ )/ ﻣﹸﺮﺳِﻠﻮ‬which is a pronoun, tells that searched for, we adopted the concept of utilizing different levels of morphological knowledge. The least degree of the whole word is sound masculine plural, which relationship is the strongest between the original word and affects the whole sentence. the alternatives. One of the examples of how the determiners benefits In the Zero Level the same and identical word is the semantic analysis is the prefix (/‫/ ﺍﻝ‬l)3 which tells considered in the search process. that the word following is a definite noun. Furthermore the state of indefinite and dual is which Level One: can extracted from the morphology can also A. For the verbs: verb inflections referring to the semantically be considered as a source if semantic same tense. (e.g. ‫)ﺿﺮﺑﺘﻢ /ﺿﺮﺑﻮﺍ /ﺿﺮﺏﹶ‬ knowledge such as unique quantification. Another B. For the derivative nouns: its grammatical states example is the prefix (/ ‫ ,/ ﺱ‬s) that direct the meaning (‫.)ﻗﺎﺋﻞٌ، ﻗﺎﺋﻼﹰ، ﻗﺎﺋﻞٍ، ﺍﻟﻘﺎﺋﻞُ، ﺍﻟﻘﺎﺋﻞَ، ﺍﻟﻘﺎﺋﻞ‬ of present verb to future. C. For the gerund: its grammatical states ( ،ً‫ﻗﻮﻝٌ، ﻗﻮﹾﻻ‬ The Morphological Pattern of a word ٍ‫.)ﻗﻮﹾﻝ‬ It is true, even not clear for every one, that the patterns of Arabic words include syntactic and semantic Level Two: content. The pattern (/َ‫ )/ﻓَﻌﹸﻞ‬of the past verb tells that the A. For the verb: verb inflections referring to all verb is intransitive which means that the sentence is tenses. (e.g. ‫)...ﺗﻀﺮﺑﺎﻥ /ﺍﺿﺮﺏ /ﻳﻀﺮﺏ /ﺿﺮﺏﹶ‬ complete with no need to have an object. An example B. For the Derivative noun: its grammatical and of the semantic profit is the meaning of request that morphological states ( ٌ‫ﻗﺎﺋﻼﺕﹲ، ﻗﺎﺋﻠﲔ، /ﻗﺎﺋﻼﻥ /ﻗﺎﺋﻠﺔ ٌ /ﻗﺎﺋﻞ‬ given by the pattern (/‫ )/ﺍﺳﺘﻔﻌﻞ‬and its derivatives. ِ‫.).ﻗﺎﺋﻼﺕٍ، ﺍﻟﻘﺎﺋﻠﻮﻥ، ﺍﻟﻘﺎﺋﻼﺕﹸ، ﺍﻟﻘﺎﺋﻼﺕ‬ The root C. For the gerund: its grammatical and The root: All roots have a semantic aspect because the morphological states (ِ‫.)ﻗﻮﻝٌ، ﻗﻮﹾﻻً، ﻗﻮﹾﻝٍ، ﺍﻟﻘﻮﹾﻝُ، ﺍﻟﻘﻮﹾﻝَ، ﺍﻟﻘﻮﹾﻝ‬3 Level Three: Please notice that deep semantic analysis considering thelogical compositionality of determiners such as (/‫، ,/ ﺍﻝ‬l) is out of A. Connecting the verb only to its gerund and vice versa (e.g. ‫.)ﺍﻟﻀﺮﺏِ /ﺿﺮﹾﺏ /ﺿﺮﹶﺏﹶ‬scope of this paper. Determiners in the logical sense (Haddad B.2007) represent quantifiers and a special case of GeneralizedArabic Quantifier (GAQ) such as: B. Connecting the derivative noun to its corresponding morphological counterparts that⎡( /‫ , /ﻣﻌﻈﻢ-ﺍﻝ‬most-of -the)⎤ share the same morphological category. For⎢ ⎥ ≡ λR.λS.( /‫(ﻣﻌﻈﻢ-ﺍﻝ‬x)/, most-of-⎢CAT DETArab⎣ ⎥ ⎦ sem Example connecting exaggeration patternthe(x) ). (R, S), where |R ∩ S| > |R –S| , whereas and in together such as (... ‫)/ﻗﻮﹼﺍﻝٌ / ﻣِﻘﻮﺍﻝ‬this context (/‫، ,/ ﺍﻝ‬l) can be regarded as a numeric Level Four:quantifier: A. Connecting the verb to its gerund and its derivative nouns and vice versa (e.g. ‫ﻣﺴﺘﻘﻴﻞ /ﺍﺳﺘﻘﺎﻝ‬⎢ ( /‫, /ﺍﻝ‬The)⎡ ⎤ ⎥ as ⎡(/‫ , / 1 ﺍﻝ‬The1 )⎤ / ‫.)ﻣﺴﺘﻘﺎﻝ‬⎢ ⎥⎢ CAT DET ⎥ ⎣ ⎦sem where⎢⎢ AG ⎡ NUM sing ⎤ ⎥ ⎦ ⎥sem⎣ ⎣ ⎦ B. Connecting the gerunds and the derivative nouns⎡(/‫ﺍﻝ‬ to each other ( ‫.)ﺍﺳﺘﻔﺎﺩﺓ /ﻣﺴﺘﻔﺎﺩ /ﻣﺴﺘﻔﻴﺪ‬⎣ 1 /, The1 )⎤ ⎦ sem ≡ λP. λQ .∃x.(∀y (P(y) ⇔ x = y) ∧ Q(x)) 160
  3. 3. following: And finally there is the Root Level, representing the most possible search in the context of semantic In the same level of (/‫ ,/ ﺿَﺮﹶﺏﹶ‬hits) is the only suggested dependency. alternative. In next level or the first one, the suggested alternatives are3. Enhancing Search based on different level of Morphological Knowledge ( ‫ …ﺿَﺮﹶﺑﹾﺖﹶ /ﺿَﺮﹶﺑﹶﺎ /ﺿَﺮﹶﺑﹾﺘﻢ /ﺿَﺮﹶﺑﻮﺍ /ﺿَﺮﹶﺏﹶ‬etc.)To enhance the quality of results by minimizing the The next level would suggest alternatives such as ( ‫/ﺿﺮﹶﺏﹶ‬number of results while maintaining more accuraterelevancy and precision, a hybrid approach combining ‫…ﺍﻟﻀﺮﺏِ /ﺿﺮﹾﺏ‬etc.)morphological analysis and generation was applied. Inthis approach a query goes through the following In next level the suggested candidates are ( ‫/ﻳﻀﺮﺏ /ﺿﺮﺏﹶ‬processes: ‫...ﺗﻀﺮﺑﺎﻥ /ﺍﺿﺮﺏ‬etc.) Morpho-Syntactic analysis to define its root/stem and part of speech. In fourth level the suggested alternatives are ( ‫/ﺿﺎﺭﺏ /ﺿَﺮﹶﺏﹶ‬ Morphological generation to produce the nearest ‫…ﻣﻀﺮﻭﺑﺘﻴﹾﻦ /ﻣﻀﺮﻭﺏ‬etc.) inflections to that word based on a semantic categorization of the different patterns that may In last level the suggested alternatives are all the nouns apply to that root/stem. and verbs that have the root (‫.)ﺿَﺮﹶﺏﹶ‬When you search based on a word’s root, you are sure toget all of the relevant results, but with other irrelevantones. Because although each result retrieved includes atleast one inflection of the word, this inflection might not 4. Conclusion and Outlookbe necessarily relevant. In this paper, we have tried to stress on the importance ofThis feature of Arabic language can not be controlled utilizing different level of morphological knowledgewithout having a lexicon that stores semantic knowledge. towards improving the quality and quantity of the AddaallIn that case, all the words will be defined with their Arabic Search engine. The results are very promising assemantic relationships to other words in the lexicon, the coverage and the sensitivity of this approach isextending the possibilities to cover not only inflections relatively incredibly practical (see link to the Searchbut synonyms and even related conceptual and ontological Engine Furthermore, our approach has considered the concept ofIn the case of stem search, stemming techniques for categorizing Arabic patterns according to their meanings,Arabic are not very well defined until now. The definition whereas some semi-semantic information has been usefulof a stem in Arabic is often exchanged with that of the in the search process. In this context, we have establishedroot. Many researchers even argue that if the stem has a a matrix correlating each pattern of speech to semanticallydifferent meaning than a root then it does not apply to related pattern. The different levels of morphologicalArabic language. They consider it a foreign concept. knowledge can be considered as an incremental process considering the next sensitive word pattern; i.e. to in-When applying stemming to an Arabic word, we mean crease the recall, and as a predication and heuristic proc-that we eliminate or remove prefixes and suffixes from ess to increase the precision; as heuristically related wordthat word. What is left is a partial word that in many might occurred in a near distance.cases has possibly no meaning and does not exist in thedictionary as it is. We then use this partial word to do Finally, although we understand that this approach pro-partial search looking for words that include this part. duces a relative and not complete set of results, we can see that it in fact represents a significant improvementThe outcome of such a search has fewer results than the towards Arabic search engines in view of its practical andsearch by root. But how significant are these in term of pragmatic use in real implementations.precision and recall? That is why we believe that Arabicstem search is neither accurate nor comprehensive.This led us to suggest a method to improve Arabic search Referencestechniques by using the morphological generator. In orderto do so, we classified the morphological patterns of Abuleil S., Alsamra K. (2004). New Technique to SupportArabic semantically according to their meanings. For each Arabic Noun Morphology: Arabic Noun Classifierword we want to search for, the most morphologically and System (ANCS). International Journal of Computersemantically related word form will be generated to be a Processing of Oriental Languages, Vol. 17 Issue 2.subject of the improved search process. Aitao Chen and Fredric Gey (2002). Building an ArabicAn Example: stemmer for information retrieval. The Eleventh TextIf the user is searching for (/‫ ,/ ﺿَﺮﹶﺏﹶ‬hits) then we notice the Retrieval Conference (TREC 2002), National Institute of Standards and Technology (NIST). 161
  4. 4. Aljlayl M. (2002). Arabic Search: Improving the Retrieval Effectiveness via a Light Stemming Approach. ACM Eleventh Conference on Information and Knowledge Management.Brin S. and Page L. (1998). The anatomy of a large hypertextual web search engine. Web publication, Stanford University.Haddad Bassam (2007). Semantic Representation of Ara- bic: A Logical Approach towards Compositionality and Generalized Arabic Quantifiers. International Journal of Computer Processing of Oriental Lan- guages, Vol. 20. Nr. 1. 2007 162