With the rise of social media, writings by ordinary people are becoming increasingly available for linguistic analysis. Such analyses offer great opportunities to identify individual users' needs from user-generated content, so that better tailored products or services can be recommended. Literature suggests that several types of human needs are universal and directly influence consumer purchase behavior. In this paper, we investigate the use of social media to identify such fundamental needs for individuals. We developed psychometric measures of universal needs through a crowd-sourced study. We also built several models to predict people's needs based on their writings. We conducted a detailed analysis of the models and showed that our models can effectively identify users' needs based on their social media data. Our results also confirm that some inferred needs correlate well with the actual product purchases and suggest a great potential for our models to significantly increase effectiveness of product recommendations.

1. 1
Contact: Huahai Yang
huahai.yang@gmail.com
Towards Computational Discovery of
Needs from Social Media

2. 2
The Buzz of the Crowd
• Hundreds of millions of
people express themselves
on social media daily
2
•500+ million tweets daily
•3.2 billion Facebook likes
and comments daily

3. 3
Analytics of Aggregates
Monitoring and Reporting
Analytics of Individuals
SentimentSentiment
ListeningListening EngagementEngagement WorkflowWorkflow
MeasurementMeasurement
PublishingPublishing
Net PromoterNet Promoter
Network TopologyNetwork Topology
Personal TraitsPersonal Traits
What are people saying? How
information is propagated?
How do people feel about my msg?
What are the likely events?
Who is this individual like? What
does she value ? What motivates
her? What is her taste and style?
Social GenomeSocial Genome
DemographicsDemographics
Insights from Social Multimedia
[Gilbert & Karahalios '09, Golbeck et al. '11, Pennacchiotti & Popescu '11]
Mass
Individual

4. 4
Discovering Personal Traits
• Basic traits
– Personality (Yarkoni, 2010)
– Human basic values (Chen et al. 2014)
– Fundamental human needs
– Emotional style
– Cognitive style
• Composite traits
– Trust
– Resilience
Personal traits are psychological, affective, and cognitive
characteristics that uniquely identify one individual from
another

5. 5
Discovering Basic Personal Traits: Our Methodology
Predicative
Models
Personal Traits
Social Media Posts
Fundamental
Needs
…
Large-scale psychometric
studies
Learning Models
Derivation of psycholinguistic
and physiognomic evidences
“Lexicons”



Prediction of personal traits

6. 6
Discovering Fundamental Human Needs
[Ford 2005]
• Fundamental needs are
universal and hierarchical
[Maslow 1943, Aaker
1995, ]
• Often change with life
events
• Guide actions and
decision making
• Brand/product choices
• Occupational choices
• Motivation
• . . .

7. 7
Challenges in Deriving Fundamental Needs
• No standard tests available
• Need to develop our own psychometric scale to measure
needs
– Using psychometric method
– Initial item pool based on literature
• 10 items each for 12 needs
– Item selection is non-trivial
Example of an “item” for “Ideals” dimension

8. 8
Item Selection
• Preliminary testing of initial item
pool on
– 360 respondents from general
public
• Examine response distributions of
individual items
– Eliminate items with highly
skewed dist.
– Retain items showing a broad
range of dist.
• Structural analysis for each need
– Ensure average inter-item
correlations fail in [.15, .5]
– Ensure internal consistency
(Cronbach’s alpha > .7)
• Retain four items after
two rounds of tests
(n=720)
• Their average score fits a
normal distribution
An Example for “Ideals”
I have a yearning for glamour and sophistication.
I crave refinement in life.
I deeply appreciate anything that strives to reach the
seemingly unattainable ideals.
I tend to pursue and worship perfection.

9. 9
Collecting Crowd’s Textual Description of Needs
• “Please describe three things that you want to get or need to do the most,
and explain why you want or need them. Please be as honest as
possible.”
• Minimal requires 60 words, average written 103 words
• Some examples:

10. 10
Predicting Needs Scores from Text: TextBasic Model
• Feature: tf-idf
• Model: Elastic-net regularized generalized linear model (Freideman et al. 2010), for
regression , solve:

11. 11
Four Models
• TextBasic
– Unigram
– No stemming
– No stop words removal
• TextExpanded
– Synonyms expansion
– Using WordNet
• TextFiltered
– Only consider sentences that indicate needs
– Using Stanford Dependency Parser to identify such sentences
• TextExpandedFiltered

12. 08/19/14 12
Build Needs Models
• Crowd-sourced needs scores and text descriptions from 2587
people on MTurk
• Models generate 12 dictionaries for each need
– 10-fold cross validated
• Prediction for certain needs is better than others
– Closeness, Ideal, Excitement, Harmony, Challenge

13. 08/19/14 13
Example Words Learned
• Some words make intuitive sense
– “family”, “love” -> positive closeness
– “your”, “critical” -> negative closeness
– “college”, “gym” -> positive excitement
– “litterbox”, “doctor” -> negative excitement
• Some words do not
– “handmade” -> negative closeness?

14. 14
External Validity: Predict Purchases by Derived Needs
• Food 2005 prescribes the associated
purchase with each need
• Test how well our derived needs
match the prescription
1. Identify Twitter users who tweeted about having bought these products,
call them Buyers
2. Derive a need for two groups of Twitter users:
• Buyers (100)
• Random Twitter users (1000)
3. Measure how good the needs models are

15. 15
Measure Model Prediction: Lift
• A performance measure used in direct marketing
– Rank the people according to the model prediction
– Si is the number of Buyers in the ith decile
• Lift Index
– 50% - chance model
– 100% - optimal model

16. 16
Results
Buyer/non-buyer classifier
trained using tweets text directly
• For the best model of Ideal need, if we
market only to the top 20 percent of
people, the expected lift almost doubles
that of a random model (40% vs. 20%)
Lift Index Model of Ideal need

17. 17
Potential Applications
• Individualized targeting based
on personal traits
– Information solicitation (Q&A)
– Information spreading
– Task solicitation
• Individualized self-branding,
interaction, or intervention
– Customer engagement
– Employee engagement
– Match-making (e.g., patient-
doctors)
NeedsNeeds Ideals: high
Recommend fine
high end products.

18. 18
Summary
• It is possible to model and automatically derive the
personal traits of individuals from social multimedia
content
• Potential use of the derived traits of individuals to
deliver hyper-personalized engagements
• Much left to be done!