Social media monitoring involves collecting and analyzing data from social media channels to track brand mentions, understand consumer sentiment, and identify opportunities. This requires acquiring data through APIs or scraping, storing large volumes of streaming data, cleaning unstructured text data using machine learning, classifying text by topic or sentiment, and visualizing insights. Key challenges include word segmentation, named entity recognition, removing unrelated comments, and topic modeling for large amounts of social data from multiple sources.

2. Challenges of NLP for
Social Media Monitoring
Parinya Sanguansat

3. What is Social media monitoring?
• Handle all the individual messages across your social channels
• Measure and analyze these data
• All of them have a dashboard
• Some of them have an action center
• Many terms: Monitoring, Listening, Understanding

4. Why we need to monitor Social media?
Passive
• Alert system
• Know yourself
• Know your competitors
• Find opportunity
• Find influencers
Active
• All in 1 Action center
• Tracking your campaign

5. Data Acquisition
• How?
• Official API
• Web scraping

6. Data Acquisition
• What?
• Keywords?
https://help.brand24.com/article/27-how-do-i-use-required-excluded-keywords

7. Data Storage
• In-house
• Own brand and their competitors
• Third party
• All data
• Database
• Huge volumes of data
• Streaming data
• Variety
• Fast query

8. Data Storage
• Relational or Document-oriented?
https://developer.couchbase.com/documentation/server/3.x/developer/dev-guide-3.0/compare-docs-vs-relational.html

9. Data Cleaning
• Clean by experts
• Clean by ML
• Challenges
• Word segmentation
• Optimum period
Data
Label valid data
by experts
Valid Data
Invalid
Data
ML

10. Word Segmentation & Named Entity Recognition
• Dictionary based
• ML-based
• Conditional Random Fields (CRF)
• Recurrent Neural Network (RNN)

11. Word Segmentation & Named Entity Recognition
Word segmentation
Named Entity Recognition
บน|ถนน|บาง|กรวย|
BI BII BII BIII
บน|ถนนบางกรวย|
OO BIIIIIIIII
• Merging
WS NER
บน|ถนน|บาง|กรวย|
O B I I
N N N
บน|ถนนบางกรวย|
BI BIIIIIIIII
OO NNNNNNNNNN
WS+NER

12. Text Categorization
• Supervised
• User-defined class
• Unsupervised
• Topic modeling
• Feature
• Frequency
• Tf-idf
• Embedding
• Character
• Sub-word
• Word
• Sentence or paragraph
• Classifier
• SVM
• NB
• Tree
• NN

13. The new state-of-the-art
http://www.wildml.com/2015/11/understanding-convolutional-neural-networks-for-nlp/

14. Sentiment Analysis
• Supervised training
• Use polar words
• How to find polar words? (From WordVector)
• Unsupervised
• Transfer learning
https://github.com/NVIDIA/sentiment-discovery

15. NLP in Engagement rate
• An engagement rate is a metric that measures the level of
engagement that a piece of created content is receiving from an
audience. (Like, Share, Comment)
• Some comments are not related to content
• Disable user post on Facebook page, user will reply in others
• Topic detection

16. Data Visualization
• Quantity
• Text (>400)
http://textvis.lnu.se/