The document discusses sentiment analysis of Arabic food delivery service reviews. It introduces the topic and motivation for analyzing sentiment to help businesses improve customer satisfaction. The goal is to apply sentiment analysis to Arabic reviews to better monitor feedback. It describes preprocessing the dataset, including cleaning, normalization, stemming and part-of-speech tagging. Support vector machines and decision tree classifiers are trained and evaluated on the preprocessed dataset, showing improved performance over the raw data. Negative reviews are analyzed to identify common complaints around wait times, food quality and billing issues.

1. ARABIC SENTIMENT ANALYSIS
O F F O O D DEL IVERY SERVICES
REVIEWS
DR. DHEYA MUSTAFA
FACULTY OF ENGINEERING
THE HASHEMITE UNIVERSITY
Oct 24, 2023 ISNCC’2023, Doha, Qatar 1

2. OUTLINE
•Introduction
•Motivation
•Goal
•Contribution
•Proposed Model
•Performance Evaluation
•Conclusion
2

3. INTRODUCTION
• Food delivery services (FDSs) have
introduced diversity to the increasing
demand for online food delivery
marketplaces.
• Mostly third-party marketplace apps for
global ordering and delivery, such as
UberEATS, Talabat (the Arabic word for
orders), and Menulog.
• Employ a cost intensive, aggregator
business model and are in charge of all
delivery logistics. 3

4. MOTIVATION
• The majority of businesses want to
effectively increase customer
satisfaction by using data to identify
areas for improvement.
–Mostly based on reviews from
customers.
• Sentiment analysis (SA) can
determine the customers’ opinion
based on the written content.
4

5. GOAL
• Apply SA to Arabic content to give an even
better service, driven by the feedback of
customers.
• Offer a quick and efficient monitoring
system to handle the needs clients on a
large scale at a low cost and maximum
profit.
5

6. METHODOLOGY
• gathering Arabic reviews regarding FDS and
evaluating feelings using popular ML methods.
• Created an Arabic dialects lexicon including
various dialects and their standard orientation
in Arabic.
• Emotions are identified and analyzed using an
existing emotion lexicon.
• experiments were carried out before and
after applying cleaning and preprocessing .
• Negative sentiments were analyzed to identify
the possible causes. 6

7. 7

8. DATASET CLEANING
• The raw datasets contain many empty and repetitive reviews that
should be efficiently eliminated.
• Involves removing duplications, identifying emojis, characters,
and word replacements, spelling correction, and annotation.
• SA of dialectical Arabic is challenging.
• We constructed a dialects dictionary manually to translate
dialectical phrases into equivalent MSA in the context of FDS.
• Popular English words written in the Arabic alphabet in the
reviews are included in the dialects dictionary as well.
– Example, the word “nice” is written as ‘‫نايس‬ ’ and is equivalent
to the word ‘‫جميل‬ ’ in Arabic.
8

9. 9

10. ARABIC REVIEWS
PREPROCESSING
• Clean review texts are process using common NLP
tasks.
• normalization : represent step characters that have
more than one form like ( ،‫أ‬
،‫إ‬
‫آ‬ ) in a unified form.
• Also removed any punctuations, diacritics, and tatweel.
• Stopword removal: reduce the size of the text by
removing unnecessary particles and pronouns.
• Tokenization: split the text into single units such as
sentences and then into single words.
10

11. ARABIC REVIEWS
PREPROCESSING
• Part of Speech (PoS) tagging: annotate words in a text
based on their type and their relationships to
neighboring and related words in that text.
– Based on the Stanford tagger [35].
• The last NLP task involved removing suffixes or
prefixes.
– To efficiently convert each word in its base form without
extensions (Stemming)
– Used the tashaphyne stemmer
11

13. SENTIMENT
CLASSIFICATION
• The final phase was to build and train the machine
learning (ML) classifier using a labeled subset of
the dataset (Talabat FDS ).
• We tested the model on unseen reviews and
categorized them into positive, negative, or
neutral classes.
• ML models: DT and SVM
– Most frequently used classifiers in Arabic SA
– Often provide the best results over other ML classifiers

14. EVALUATION
14

15. EVALUATION
15
Experiment
DT classifier SVM Classifier
P% R% F% Acc% P% R% F% Acc%
Before
cleaning
and pre-
processin
g
+ve 67 95 79 87 86 86
-ve 82 44 57 80 86 83
Neu 20 4 6 31 23 26
Avg 68 69 65 69 79 80 80 80
After
cleaning
and pre-
processin
g
+ve 72 91 81 87 88 88
-ve 76 60 67 77 89 83
Neu 20 3 5 33 10 15
Avg 69 73 69 73 79 82 80 82
+ve: positive -ve: negative Neu: neutral Avg:
average

16. EVALUATION
16

17. EVALUATION
17
Fig. 3. A confusion matrix of the SVM classifier after applying cleaning
and preprocessing phases.

18. EVALUATION
18

19. CONCLUSION
• Proposed a sentiment analysis model of Arabic dialect
reviews about FDS.
• applied intensive preprocessing and several aspects of
analyzing Arabic text to generate a dataset.
• examined two well-known ML classifiers over this
dataset and provided a comparative performance
evaluation
• Analyzed the most representative negative reviews
and produced a word cloud.
– The long waiting time, cold food, wrong order, and
bill were the most important factors contributing
to the negative sentiments.
19

20. Thank you
Oct. 23, 2023 ISNCC'2014, Doha, Qatar 20