NLPMeetup#2 @PIM

2. Challenges of NLP for
Social Media Monitoring
Parinya Sanguansat

3. What is Social media monitoring?
• Handle all the individual messages across your social channels
• Measure and analyze these data
• All of them have a dashboard
• Some of them have an action center
• Many terms: Monitoring, Listening, Understanding

4. Why we need to monitor Social media?
Passive
• Alert system
• Know yourself
• Know your competitors
• Find opportunity
• Find influencers
Active
• All in 1 Action center
• Tracking your campaign

5. Data Acquisition
• How?
• Official API
• Web scraping

6. Data Acquisition
• What?
• Keywords?
https://help.brand24.com/article/27-how-do-i-use-required-excluded-keywords

7. Data Storage
• In-house
• Own brand and their competitors
• Third party
• All data
• Database
• Huge volumes of data
• Streaming data
• Variety
• Fast query

8. Data Storage
• Relational or Document-oriented?
https://developer.couchbase.com/documentation/server/3.x/developer/dev-guide-3.0/compare-docs-vs-relational.html

9. Data Cleaning
• Clean by experts
• Clean by ML
• Challenges
• Word segmentation
• Optimum period
Data
Label valid data
by experts
Valid Data
Invalid
Data
ML

10. Word Segmentation & Named Entity Recognition
• Dictionary based
• ML-based
• Conditional Random Fields (CRF)
• Recurrent Neural Network (RNN)

11. Word Segmentation & Named Entity Recognition
Word segmentation
Named Entity Recognition
บน|ถนน|บาง|กรวย|
BI BII BII BIII
บน|ถนนบางกรวย|
OO BIIIIIIIII
• Merging
WS NER
บน|ถนน|บาง|กรวย|
O B I I
N N N
บน|ถนนบางกรวย|
BI BIIIIIIIII
OO NNNNNNNNNN
WS+NER

12. Text Categorization
• Supervised
• User-defined class
• Unsupervised
• Topic modeling
• Feature
• Frequency
• Tf-idf
• Embedding
• Character
• Sub-word
• Word
• Sentence or paragraph
• Classifier
• SVM
• NB
• Tree
• NN

13. The new state-of-the-art
http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

14. Sentiment Analysis
• Supervised training
• Use polar words
• How to find polar words? (From WordVector)
• Unsupervised
• Transfer learning
https://github.com/NVIDIA/sentiment-discovery

15. NLP in Engagement rate
• An engagement rate is a metric that measures the level of
engagement that a piece of created content is receiving from an
audience. (Like, Share, Comment)
• Some comments are not related to content
• Disable user post on Facebook page, user will reply in others
• Topic detection

16. Data Visualization
• Quantity
• Text (>400)
http://textvis.lnu.se/