An intelligent algorithm that uncovers opinions about trends, campaigns, and brands on Twitter. DORA provides the sentiments that surround a certain theme, whether positive, negative or neutral. Check this Twitter tool here: https://www.dora-socialreader.com and ask for a free trial.

1. SENTIMENT
ANALYSIS
ON SOCIAL MEDIA | ASDARS_

2. ASDARS_
Big Data Sentiment Analysis NER

3. igh Volume_
adoop_
VOLUME
VELOCITYVARIABILITY
BIG DATA_
H

4. TWITTER_ ❤ |
DATA FILTERS
 Hashtags
 Metadata
 Influencers
BAD DATA
 2016 Presidential Campaign
 Cynk (How a company with one employee, no assets, and
$0.00 revenue, got a $6 billion market value and blew up)
 BotOrNot

5. ASDARS_
• TWITTER
• Blogs
• Reviews
• Facebook (…)
Sources
• Hadoop
• Machine
Learning
Processing
• Web
• Mobile
API

6. BIG DATA, THE FIRST PIPELINE_
Flume Oozie HDFS
 Flume: Sources, Sinks, Channels
Twitter
Source
Memory
Channel
HDFS Sink
 Hive: Distributed Storage, SQL
 Oozie: Job Scheduler
HDFS SerDe Hive
Hive

7. WHAT WENT WRONG_
 Virtualized Infrastructure
 Hive: poor performance
 SQL queries very limited
Flume Oozie HDFS Hive

8. BIG DATA, THE SECOND
PIPELINE_
 Tweepy: a Python Twitter API client. Publishes to Kafka.
 Kafka:
 Spark:
 MongoDB
 Flask
Tweepy Kafka Spark MongoDB Flask

9. MACHINE LEARNING_
 Classification Algorithms: polarity detection
 Naive Bayes, SVM, SGD, Random Forests
 Labeled datasets

10. TRAINING_
 Clean Up: remove html characters; remove stopwords
 Word Stemming
 Unigrams and Bigrams
 Bag of Words/TF-IDF

11. EVALUATION,
TRAIN/TEST 80/20_
CONFUSION MATRIXROC CURVE

12. RESULTS_
ALGORITHM PRECISION %
Random Forest 78
Naïve Bayes 77
Linear SVM 80
SGD 78

13. MODEL UPDATE_
Hive OOV Classify
New
Data
UpdateEvaluateTrainLabel

14. NEXT STEPS_
Bag of Words
_Word Embeddings
Social Kafka Spark FlaskMongoDB

15. CONTACTS_
JOÃO CUNHA
Innovation and R&D CTA
jcunha@m-profiler.com
(+351) 916079254