This talk at Data Science Seminar of SSA presents challenges and methods to model behavior on social media & Web for application opportunities for public services. The talk also demonstrates an in-depth case study of mining intentional behavior from the noisy natural language text of social media messages during disasters and how it could assist emergency services of future smart cities.

1. SOCIAL MEDIA & WEB MINING
FOR PUBLIC SERVICES OF SMART CITIES
@hemant_pt
Hemant Purohit, Ph.D.
Humanitarian, Semantics & Informatics Lab
Dept. of Information Sciences & Technology
Data Science Technology Exchange Series
Analytics Center of Excellence (ACE)
U.S. Social Security Administration Headquarters, MD
Aug 13, 2018 hpurohit@gmu.edu

2. Outline
¨ Social Media & Web Data
n Opportunity to Mine Relevant Human Behaviors
n Behavior Modeling: Challenges, Types, Methodology
¨ Case study of Disaster Domain
n Mining Relevant Information
n Text Classification: Topic, Sentiment-Emotion, Intent
n Intent Mining: Supervised & Transfer Learning
¨ CitizenHelper: Social & Web Analytics System
n Natural Disasters, Violence & Stereotyping, Displacement
2

3. 21st Century Information Exchange
3
Connecting Humans at Scale!

4. Source: http://www.go-gulf.com/blog/online-time/
We cannot Ignore Web Data Anymore!
4

5. 5
Source: http://www.go-gulf.com/blog/online-time/

6. Types of Social Media
¨ Publishing
¤ Blogging
¤ Wiki
¨ Micro blogging
¨ Social News
¨ Social Bookmarking
¨ Media Sharing
¤ Video Sharing
¤ Photo Sharing
¤ Podcast Sharing
¨ Opinion, Review, & Ratings sites
¨ Answers
¨ Entertainment
6
Source: http://socialmediamining.info

7. Social Data: Big Human Interaction Data
7
Uni-directional communication
(TO people)
Bi-directional
(BY people, TO people)
Web 2.0
Social
media

8. Mining Social & Web Data: Innovation
Opportunity for Public Services
8
But ..
WHAT & HOW
to leverage?

9. Variety of Behaviors to Model
¨ User-User (link generation)
¤ befriending, sending a message,
playing games, following, or inviting
¨ User-Community
¤ joining or leaving a community,
participating in community discussions
¨ User-Entity (content generation)
¤ writing a post, posting a photo
9
Source: http://socialmediamining.info

10. Goals of Behavior Modeling
10
Source: https://www.slideshare.net/darintocommIT/how-to-use-social-media-to-fine-tune-your-communication-plan-ierrepaulfares/10
IBM Social Media Analytics Framework

11. Goals of Behavior Modeling
11
Source: https://www.slideshare.net/darintocommIT/how-to-use-social-media-to-fine-tune-your-communication-plan-ierrepaulfares/10
IBM Social Media Analytics Framework

12. Goals of Behavior Modeling
12
Source: https://www.slideshare.net/darintocommIT/how-to-use-social-media-to-fine-tune-your-communication-plan-ierrepaulfares/10
IBM Social Media Analytics Framework

13. Dimensions of Behavior Modeling:
People (User), Content, Network
13
[Sheth et al. (ESNAM 2017). http://wiki.knoesis.org/index.php/Twitris]

14. Behavior Modeling: Social Data Challenges
1. Big Data Paradox
1. Social media data is big, yet not evenly distributed.
2. Often little data is available for an individual
2. Obtaining Sufficient Samples
1. Are our samples reliable representatives of the full data?
3. Noise Removal Fallacy
1. Too much removal makes data more sparse
2. Noise definition is relative and complicated and is task-dependent
4. Evaluation Dilemma
1. When there is no ground truth, how can you evaluate?
14
Source: http://socialmediamining.info

15. Behavior Modeling: Social Data Challenges
15
Source: http://www.aolteanu.com/img/interests.jpg

16. Behavior Modeling: Basic ‘1-size-fits-all’ a.k.a.
‘general’ Solutions May Not Be Sufficient!
16
Source: http://www.gdprtoons.com/2017/09/gdpr-profiling-automated-processing-and.html ;
https://digital.gov/2014/05/07/analyzing-search-data-in-real-time-to-drive-decisions/

17. Behavior Modeling: Efficient Solutions with
Domain-specific Modeling
17
Application
Behavioral
Needs
Relevant
Data
Access
Techniques
Need of solutions at
the intersection!

18. Behavior Modeling: Illustration for Deeper Analysis:
Sentiment in Social Media User Activities
18
Multi-
faceted
Analysis
Source: https://www.ibm.com/communities/analytics/watson-analytics-blog/data-discover-and-display-the-new-watson-analytics-user-experience/

19. Help Offering
Emotionally
Supporting
Expertise
Sharing
Joking
Marketing
Manipulating
Harassing
Bullying
Propagandizing
Deceiving
Rumoring
Accusing
Sensationalizing
Social Good Social Bad Social Ugly
Positive
Effects
Negative
Effects
Behavior Modeling: Illustration for Deeper Analysis:
Intent behind Social Media User Activities
[Purohit & Pandey (LNSN 2018, in press). Intent Mining for the Good, Bad & Ugly Use of Social Web: Concepts, Methods, and Challenges.]
19

20. Behavior Modeling: Methodology
¨ Analyze or predict behaviors
¤ user, content, network or hybrid dimensions
¤ Users exhibit different behaviors on social media:
n As individual user, or
n As part of a broader collective ‘group’ behavior.
n Collective behavior emerges when a population of individuals behave
in a similar way with or without coordination or planning.
20
Source: http://socialmediamining.info

21. Behavior Modeling: Methodology Focus
21
[Sivarajah et al. (JBR 2017). https://www.sciencedirect.com/science/article/pii/S014829631630488X]

22. Behavior Modeling: Methodology Types
¨ Predictive Tasks
¤ Use some variables to predict unknown or future values of other variables
n Classification
n Regression
n Deviation Detection
¨ Descriptive Tasks
¤ Find human-interpretable patterns that describe the data
n Clustering [Descriptive]
n Association Rule Discovery [Descriptive]
n Sequential Pattern Discovery [Descriptive]
22
[Han et al. (2011). Data Mining: Concepts and Techniques]

23. Behavior Modeling: Methodological Steps
Set an
observable
behavior
The behavior
needs to be
observable, e.g.,
accurately
observing the
joining of
individuals to
communities (and
possibly their
joining times)
Feature
Extraction
Finding data
features
(covariates) that
may or may not
affect (or be
affected by) the
behavior
[*We need a
domain expert
for this step*]
Feature-
Behavior
Association
Find the
relationship
between features
and behavior
e.g., use decision
tree learning
Evaluation
The findings are
due to the
features and not
to externalities.
e.g., we can use
classification
accuracy,
randomization
tests, or causality
testing algorithms
23
Source: http://socialmediamining.info

24. Outline
¨ Social Media & Web Data
n Opportunity to Mine Relevant Human Behaviors
n Behavior Modeling: Challenges, Types, Methodology
¨ Case study of Disaster Domain
n Mining Relevant Information
n Text Classification: Topic, Sentiment-Emotion, Intent
n Intent Classification: Supervised & Transfer Learning
¨ CitizenHelper: Social & Web Analytics System
n Natural Disasters, Violence & Stereotyping, Displacement
24

25. Case Study: Disaster & Humanitarian Events
25

26. Case Study: Relevancy Challenge
¨ Relevance
challenge
¤ High-noise
¤ Low-signal
26

27. Case Study: Response Needs of Emergency Services
Source: http://www.kolotv.com/content/news/Learning-from-Hurricane-Harvey-442420813.html
https://twitter.com/abc13houston/status/936188111293440000
Fundamental Information Requirement:
WHO needs WHAT, WHERE & WHEN
Event
Response
Coordination
27

28. DISASTER
Event
Relevant Behavior to Services: Intent for Help
28
CITIZEN
Sensors
RESPONSE
Organizations
x
INTENTIONAL DATA

29. Relevant Behavior to Services: Intent for Help
29
¨ Outcome: Proactivity mitigating resource demand-supply mismatch
Hurricane Sandy, “Thanks, but no thanks”, NPR,
Jan 12 2013
http://www.npr.org/2013/01/09/168946170/thanks-but-no-thanks-when-post-disaster-donations-overwhelm

30. Intentionality
30
¨ Intent: Purpose or aim for an action
¨ Latent in the textual or verbal utterance

31. Short-text & Intent: Examples
31
¤ Social media text: unstructured, informal language, short
EXEMPLAR DOCUMENT INTENT
@NYGo corrections officers @ riker working / trapped
during hurricane haven't bn provided food, plz help
SEEKING HELP
Anyone know where the nearest #RedCross is? I wanna
give blood today2 help victims of #Sandy
OFFERING HELP
Would like to urge all citizens to make the proper
preparations for #Sandy - prep is key -
http://t.co/LyCSprbk has valuable info!
ADVISING
[Purohit et al. (IEEE SocialCom 2015). Intent Classification of Short-Text on Social Media]

32. Short-text & Intent: Problem
32
¨ Intent: Aim of action
SHORT-TEXT INTENT
@NYGo corrections officers @ riker working / trapped
during hurricane haven't bn provided food, plz help
SEEKING HELP
Anyone know where the nearest #RedCross is? I wanna
give blood today2 help the victims of #Sandy
OFFERING HELP
Would like to urge all citizens to make the proper
preparations for #Sandy - prep is key -
http://t.co/LyCSprbk has valuable info!
ADVISING
How to identify relevant intent from ambiguous, unconstrained
natural language text?
Text Classification Problem

33. Text Classification: Problem Variants
33
TEXT CLASSIFICATION
TYPE
FOCUS EXAMPLE
Topic predominant
subject matter
sports or entertainment
Sentiment/Emotion/
Opinion
focus on present state
of emotional affairs
negative or positive;
happy emotion
Intent Focus on action, hence,
future state of affairs
offer to help after floods
e.g., I am going to watch the awesome Fast and Furious movie!! #Excited
[Purohit et al. (IEEE SocialCom 2015). Intent Classification of Short-Text on Social Media]

34. Illustration for Topic Classification
34
Caution &
Advice
Information
Sources
Damage &
Casualties
Donations
Health
Shelter
Food
Water
Logistics
...
...
[Purohit, Castillo, Meier, & Sheth (ICWSM-Tutorial 2013). https://corescholar.libraries.wright.edu/knoesis/583/]

35. Intent Classification: Supervised Learning Method
35
Source: http://datasciencecheetah.com/?page_id=426

36. Intent Classification: Supervised Learning Method
36
Tweets with Class Labels
Learning Goal:
Prediction Goal:

37. Intent Classification: Challenges that motivate the
need for rich feature representation
37
¨ Ambiguity in interpretation
¨ Sparsity of relevant behaviors
n 1% signals (Seeking/Offering) in 5M tweets during #Sandy
[Purohit et al. 2013]
¨ Inefficient intent representation cues
n e.g., commercial intent, F-1 score 65% on Twitter [Hollerit et al. 2013]
I wanna help @Good with Hurricane Relief. Also plz text SANDY to 80888 & donate $10 @redcross
*Blue: offering intent, *Red: seeking intent

38. Learning
Improves
Meaning
Expressivity
Increases
BOTTOM-UP PROCESSING FEATURES
{Bag of Word Tokens}
Independent Token Vectors using
Bag-of-Words Model
(patterns derived from the data)
TOP-DOWN PROCESSING FEATURES
{Set of Patterns}
Rules using Declarative (DK) & Social Behavior
(SK) Knowledge and Contrast Mining
(patterns defined for intent association)
Creating
Rich Features for
Intent Classification
[Purohit et al. (IEEE SocialCom 2015)]

39. Intent Classification Hybrid: Bag-of-Tokens
Features
39
¨ (T) Bag-of-Words model is the simplest alternative for structured representation of text
Source: https://www.oreilly.com/library/view/applied-text-analysis/9781491963036/ch04.html

40. Intent Classification Hybrid: Rule-based Pattern
Features
40
(DK) Declarative Knowledge Patterns
● Domain expert rules and psycholinguistics syntactic-semantic rules
e.g.,
(SK) Social Knowledge Indicator Patterns
● Offline conversation indicators (often considered ‘stopwords’)
e.g., Presence of Dialogue Management cues such as {‘thanks’, ‘anyway’, ..}
Feature_Patternj (messagei) = 1 if Pattern j exists in message i , else 0
[Purohit et al. (IEEE SocialCom 2015). Intent Classification of Short-Text on Social Media]

41. Intent Classification Hybrid: Extracted Pattern
Features
41
(CTK) Contrast Knowledge Patterns
INPUT: corpus {mi} of message texts
For each label class:
● Find contrasting pattern unique to the class by using sequential pattern mining
(CPK) Contrast Patterns: on Part-of-Speech tags of messages
e.g., unique sequential patterns:
HELP-SEEKING Intent class: help .* victim .* _url_ .*
HELP-OFFERING Intent class: anyone .* know .* clothing .*
[Purohit et al. (IEEE SocialCom 2015). Intent Classification of Short-Text on Social Media]

42. CORPUS
Set of
short text
documents,
S
FEATURES
Bag-of-Tokens & Set-
of-Patterns features
XT
, y
M_1
M_2
M_K
.
.
.
Subset Xj
T ⊂ S such that, Xj
T includes
all the labeled instances of class cj for
model M_j
P(c2)
P(c1)
X1
T, y1
X2
T,y2
XK
T,yK
P(cK)
(In 1-vs-1 learning framework: K*(K-1)/2 classifiers, for each (cj,ck) class pair)
Binarization Framework for Learning:
1-vs-All
42

43. Intent Classification Hybrid: Combining Bag-of-
Tokens & Rule-based Pattern Features perform well!
Avg. F-1 Score (10-fold Cross Validation)
Frameworks:
Gain 7%, p < 0.05
Dataset-1
(Hurricane Sandy
2012)
(Declarative)
(Social)
(Contrast)
T,DK,SK,CTK,CPK
T,CTK,CPK
43
[Purohit et al. (IEEE SocialCom 2015)]

44. Intent Classification: Limits of Supervised Learning
Methods
44
¨ Acquiring a large label set for quickly training a
Supervised Classifier can be hard during a new crisis!
Source: http://knowledgeofficer.com/knowledge/46-transfer-learning-machine-learning-s-next-frontier

45. Intent Classification: Transfer Learning Method
Source: http://knowledgeofficer.com/knowledge/46-transfer-learning-machine-learning-s-next-frontier
Hurricane
Sandy Hurricane
Harvey
45

46. Intent Classification: Transfer Learning Method
46
Source: https://www.datasciencecentral.com/profiles/blogs/transfer-learning-deep-learning-for-everyone

47. Intent Classification: Transfer Learning Method
¨ Challenges
¤ Context for intent changes
n e.g., location, resource type, emergency phase
¤ Distribution of intent classes across events changes
n e.g., hurricane vs. tornado
¤ Representation of features affected by cultural nuances
n e.g., Hurricane in NYC vs. Typhoon in Philippines
47

48. Intent Classification: Transfer Learning using
Sparse Coding
48
Ø Likewise images, the expression of our intent sense comprises of many factors!
Ø Sparse Coding method inspired by vision (Olshausen & Field, 1997)
Source: https://www.cs.ubc.ca/~schmidtm/MLRG/sparseCoding.pdf

49. Intent Classification: Transfer Learning using
Sparse Coding
49
¨ Sparse coding is a method for discovering good basis vectors automatically
¤ It is similar to Principle Component Analysis but more general
Input: x(1), x(2), …, x(m) (each in Rn x n)
Learn: Dictionary of bases f1, f2, …, fk (also Rn x n), so that each input x can
be approximately decomposed as:
such that aj’s are mostly zero (“sparse”)
¨ You can think of bases as the edges in the images
¤ Likewise, for intent or behavioral cues in the text, there would be such sense cues
Read More: https://slideplayer.com/slide/4739344/

50. Intent Classification: Transfer Learning using
Sparse Coding
50
Source: https://www.cs.ubc.ca/~schmidtm/MLRG/sparseCoding.pdf
• Overcomplete basis
dictionary allows richer
representation for intent
• Improvement in F-score
and Accuracy up to 10%
[Pedrood & Purohit, SBP-BRiMS 2018]

51. Intent Classification: Transfer Learning using
Sparse Coding - Results
¨ Result column shows performance for a setting: Source Set à Target Set (x-axis)
¤ S = Hurricane Sandy, Y = Typhoon Yolanda, H = Hurricane Harvey, I = Hurricane Irma
51
Leveraging multiple event datasets as
the source shows better performance.Statistically
Significant
[Pedrood & Purohit (SBP-BRiMS 2018). Mining Help Intent on Twitter During Disasters via Transfer Learning with Sparse Coding]
Accuracy

52. Intent Classification: Application for Bipartite Intent
Matching in Context for Help Seekers & Suppliers
52
[Purohit et al. (First Monday 2013). Emergency-Relief Coordination on Social Media: Automatically Matching Resource Requests and Offers]

53. Intent Classification: Application for Prioritization
of Streaming Messages
53
¨ Intent
¤ e.g. urgent help,
volunteering
¨ Topic
¤ e.g. shelter, medical
…
Source: https://blog.bufferapp.com/twitter-timeline-algorithm
BEYOND TIME, RANK BY

54. Outline
¨ Social Media & Web Data
n Opportunity to Mine Relevant Human Behaviors
n Behavior Modeling: Challenges, Types, Methodology
¨ Case study of Disaster Domain
n Mining Relevant Information
n Text Classification: Topic, Sentiment-Emotion, Intent
n Intent Classification: Supervised & Transfer Learning
¨ CitizenHelper: Social & Web Analytics System
n Natural Disasters, Violence & Stereotyping, Displacement
54

55. CitizenHelper System: Visual Analytics of Social,
Web & Offline Data Sources
55
¨ Supports a variety of application domains:
¤ Natural Disasters
n Can we discover and rank critical help intent messages?
¤ Global displacement
n Can we predict displacement by extracting behaviors from Social &
Web streams?
¤ Gender-violence
n Can we model and explain the causes of stereotyping for specific
groups (gender, race, ..)?
¤ STEM workforce diversity
n Can we model the diversity and engagement in awareness campaigns
(e.g., #iLookLikeAnEngineer)?

56. CitizenHelper System: Addresses the Challenges
of NGO and Gov. Agency Data Analysis
¨ Surveys:
¤ Static by design
¤ Time & Resource costs
¨ Basic descriptive analysis
¤ Limited Insights for action
n Hard to reach ‘audience’
¨ Action Evaluation
¤ Qualitative frameworks
n Limited Quantification
[Purohit et al. (First Monday 2016). Gender-based violence in 140 characters or fewer: A #BigData case study of Twitter]
56

57. CitizenHelper System: Architecture
57
¨ Ingests heterogenous streaming sources
¨ Supports in-memory or web service-driven processing
[Karuna et al. (ICWSM 2017). http://ist.gmu.edu/~hpurohit/informatics-lab/icwsm17-citizenhelper.html]

58. CitizenHelper System: Social Media Visual
Analytics for Attitude Trends
• Provides contrasting analyses by information attributes (Location, Time, Topic) aids learning.
58
[Karuna et al. (ICWSM 2017). http://ist.gmu.edu/~hpurohit/informatics-lab/icwsm17-citizenhelper.html]

59. CitizenHelper System: Contrasting Analyses across
Offline and Online Data Sources
Premise: Contrasting analyses by information attributes (Location, Time, Topic) aids learning.
[Karuna et al. (2017). ICWSM]
59
[Karuna et al. (ICWSM 2017). http://ist.gmu.edu/~hpurohit/informatics-lab/icwsm17-citizenhelper.html]

60. Conclusion
¨ Social data has valuable information of user behaviors
(although needles in the haystack!) for Public Services.
¨ Efficient feature representation for learning user behavior
is crucial, and interdisciplinary research can help!
¨ Less is More! J We need to focus on actionable
behavioral information, and avoid the Big Data deluge.
60

61. TWITTER: @hemant_pt
MAIL: hpurohit@gmu.edu
Acknowledgement: image sources, sponsors, collaborators (especially Profs. Amit Sheth, Valerie Shalin, & TK
Prasad at Kno.e.sis and Prof. Carlos Castillo at UPF), HSIL students (Prakruthi, Rahul, Bahman, Cooper), and
Questions?
http://ist.gmu.edu/~hpurohit/informatics-lab.html
61
Grants: IIS #1657379 & #1815459,
DUE #1707837
Resources
on Lab site: