1
Arabic Natural Language
Processing
Nizar Habash
Columbia University
Center for Computational Learning Systems
With modif...
2
Road Map
• Introduction
• Orthography
– Arabic Script
– MSA Phonology and Spelling
– Encoding Issues
• Morphology
• Synt...
3
Introduction
• What is ‘Arabic’?
– A Semitic language
– Arabic Script
• With or without diacritics
• More ambiguity!
– A...
4
Road Map
• Introduction
• Orthography
– Arabic Script
– MSA Phonology and Spelling
– Encoding Issues
• Morphology
• Synt...
5
Arabic Script
Arabic script is an alphabet with allographic variants,
optional zero-width diacritics and common ligature...
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Habash: Arabic Natural Language Processing
Upcoming SlideShare
Loading in...5
×

Habash: Arabic Natural Language Processing

2,230

Published on

Published in: Technology, Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,230
On Slideshare
0
From Embeds
0
Number of Embeds
6
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Geared towards non-arab researchers working on Arabic NLP
    Focus on MT!
    Orthography discussion is necessary to understand later concepts and phenomena
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Letters
    Body
    Shape
    Connective
    Disconnective
    Dots
    Diacritics
    oblig/optiona;
    Vocalic/consonantal
    font
    others…
    Short/long writing
    Script feature
  • Problems for OCR … MT from OCR output?
  • This is exemplifying with a language… but in principle these symbols of a script canbe used whichever way…
  • Con / dis
    Similar shapes
    Very different forms
  • There are additional ones for koranic text that are not discussed here
  • There are additional ones for koranic text that are not discussed here
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Indefiniteness (Nunation تنوين)
    (tashdid تشديد, dagesh דגש)
  • Indefiniteness (Nunation تنوين)
    (tashdid تشديد, dagesh דגש)
  • Indefiniteness (Nunation تنوين)
    (tashdid تشديد, dagesh דגש)
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • No correspondence … standar (some programs have other additional phonbetiocv )
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Psycholinguistic reality
    format  فرمت farmat
    Dictionary ordered
    Not all combinations possible
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Article or poss
    Case
    Arabic Broken Plurals
    Intersection of Derivational and Inflectional Morphology
  • Aspect
    PA circumfix negation
    Object, iobj
  • Stems are different
    Distribution of features
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Average length of arabic words is 4.89 (5.85 unique)
    Compared to English 4.36 and 7.20
    +نا : we, us, our (subject, object, possessive)
  • Indefiniteness (Nunation تنوين)
    (tashdid تشديد, dagesh דגש)
  • My work on palestinian, arabic mt and arabic hebrew mt
    Highlight similarities and differences
    A lot of similarities/differences not included
  • Lexeme idea (UMD+columbia – me)
    Representation appropriateness for an application: Information Retrieval vs. Machine Translation
    appropriateness in terms of orthography too
  • Lexeme idea (UMD+columbia – me)
    Representation appropriateness for an application: Information Retrieval vs. Machine Translation
    appropriateness in terms of orthography too
  • Lexeme idea (UMD+columbia – me)
    Representation appropriateness for an application: Information Retrieval vs. Machine Translation
    appropriateness in terms of orthography too
  • Habash: Arabic Natural Language Processing

    1. 1. 1 Arabic Natural Language Processing Nizar Habash Columbia University Center for Computational Learning Systems With modifications
    2. 2. 2 Road Map • Introduction • Orthography – Arabic Script – MSA Phonology and Spelling – Encoding Issues • Morphology • Syntax
    3. 3. 3 Introduction • What is ‘Arabic’? – A Semitic language – Arabic Script • With or without diacritics • More ambiguity! – Arabic Language • Modern Standard Arabic (MSA) • Arabic Dialects
    4. 4. 4 Road Map • Introduction • Orthography – Arabic Script – MSA Phonology and Spelling – Encoding Issues • Morphology • Syntax
    5. 5. 5 Arabic Script Arabic script is an alphabet with allographic variants, optional zero-width diacritics and common ligatures. Arabic script is used to write many languages: Arabic, Persian, Kurdish, Urdu, Pashto, etc. PDF Files problem ‫طا‬ُ ‫ا‬ ‫خ‬َ‫ط‬ ‫ال‬ ‫ب

    ×