The document discusses quantifying the effect of social media sentiment ("buzz") on brand loyalty and customer defection for Ford Motor Company. It aims to integrate social media data, specifically from Twitter, into existing loyalty and defection models by extracting predictive signals about future vehicle purchases from customer comments. This will allow Ford to put a dollar value on salient events and intervene when negative sentiment is expressed in order to potentially improve loyalty and reduce defection.

1. Quantifying the "Buzz" Effect:
Integrating Social Media with Loyalty
& Defection Models
Marketing Associates:
Keith Shields, Director, Decision Sciences
Roni Leibovitch, Senior Consultant, Digital Intelligence
Mindy Deatrick, Senior Consultant, Quantitative Solutions
Ford Motor Company:
Margaret Kishore, Performance and Metrics Manager

2. About the Title…
 “Buzz” refers to the amount of, and sentiment of, the Ford-related comments
available through social media outlets.
 The “Buzz Effect” refers to the increase or decrease in brand loyalty / defection
(measured by repurchase) that occurs as a result of a change in the Buzz.
 “Quantifying the Buzz Effect” means we want to put a number on the amount of
that increase or decrease.
 The advantage of this is that we can begin to put a dollar value on salient, publicly-
known events…such as refusing to take government bailouts.
 “Integrating Social Media With Loyalty / Defection Models” means that we will:
 Extract signals of future vehicle purchase decisions from customer comments found
through social media outlets AND
 Capture those signals in the form of predictive variables to put into loyalty models.
 Those variables and their associated model coefficients will quantify the buzz effect.
 For the purpose of this analysis we focus our efforts on Twitter.
1

3. Warnings and Disclaimers
 We will do our best to reveal trends, patterns, and findings without
showing actual numbers (but for some cases). Hiding / changing of
numbers is done to protect the innocent (Ford Motor Company especially).
 In the course of the presentation we will share many Ford-related “tweets”.
These will be actual tweets. They will not be censored because their
informal nature highlights a point we want to make about text mining.
Please try not to be offended.
 We will not be presenting any astounding research and/or revelations
about text mining. We use “off-the-shelf” techniques when it comes to
categorizing sentiment.
 Our expertise is in modeling and predicting customer behavior based on all
available and relevant customer data.
 We see social media as a potentially rich source of customer data, and those
data just happen to be free-form text.
2

4. One More Item of Note…
3

5. Background on Ford’s Social Media Efforts…
 Measuring the “Consumer Experience”
 Alan Mulally and Apple…
 The Dealership Experience: Sales and Service
 The Ownership Experience
 How do people share experiences? Traditionally by talking to each other. But
how much today is done through Twitter, Facebook, Blogs?
 By analyzing the comments and sentiment expressed through Social Media
outlets can we glean meaningful insights about the Ford Consumer
Experience?
 Can we make inference about a consumer’s affinity for Ford…or an existing
customer’s loyalty to Ford?
 If no, then we’re probably not trying hard enough.
 Examples next 2 slides.
4

6. Google Twitter Search: Ford Comments
 Search: “My Ford Focus is great.”
 I love my Ford Focus, but not so much Ford Service in Northampton Mass.
Thieves.
 Got my new computer yesterday and can't wait to get my new 2012 Ford
Focus SEL in 4-6 weeks! 23 Apr
 Am test driving Hondas and Fords 7 Apr
 We’d like to have a mechanism for intervening here. On April 7 this person
indicated he was facing a choice between buying a Honda and buying a Ford.
 Does this mean we can simply scrape Twitter for the words “test drive”?
Seems like it would be predictive of future behavior…
5

7. Google Twitter Search: Ford Comments
 Search: “I don’t like my Ford Escort.”
 The ford escort texting and driving, I really like my life and my
car, please don't try and drive into us, twice. Close call!
 My old '93 ford escort is running 130k and runs like a charm....
And my 2003 ford ranger truck has 80k without problems.
 Again this seems like something that, if captured and
quantified in the form of a variable, would be predictive in
the context of a loyalty model.
6

8. Google Twitter Search: Ford Comments
 Search: “Ford, government bailout”
 This weekend my wife and I purchased a FORD. Why? Because they
chose not to accept the government funded bailout.
 Ford didn't accept the government bailout - that's pretty awesome.
 Wait #Ford pulled the ad that was critical of the #Obama bailout but is
now running one that jokes about drinking and driving?
 GM CEO wants higher gas tax. Buy a Ford car or truck. Please RT
 This is an example of how capturing “influencers” could be very important. This person
happens to have 340 followers and routinely tweets about auto-related topics.
 So the effort to mine Twitter for Ford sentiment extends beyond improving the loyalty
and defection models…but the title of this presentation does not. That said, we will
discuss how we are affecting marketing programs with our existing knowledge of
influencers.
7

9. Background on Ford’s Social Media Efforts…
 Measuring the “Consumer Experience”
 Alan Mulally and Apple…
 The Dealership Experience: Sales and Service
 The Ownership Experience
 How do people share experiences? Traditionally by talking to each
other. But how much today is done through Twitter, Facebook, Blogs?
 By analyzing the comments and sentiment expressed through Social Media
outlets can we glean meaningful insights about the Ford Consumer
Experience?
 Can we make inference about a consumer’s affinity for Ford…or an existing
customer’s loyalty to Ford?
 Yes! So what can we do capitalize upon good sentiment and reverse bad
sentiment?
8

10. Start With the Current Infrastructure
 The Ford Motor Company has a customer data warehouse that collects relevant data from all
customer touchpoints, “customerizes” it, and applies a suite of predictive models that are used for
targeted campaigns.
 More importantly the warehouse is connected to many customer-facing and dealer-facing
operational systems, and it passes important information about customer behavior, both past
behavior and predicted behavior, to operational systems when decisions regarding the customer
have to be made in real time.
Users & business interface Customer Data by Touch Point
Dealers Call Center Click-thru Mail Call Center
Data Creation
Sales Dealer Service
Customers Website
Survey
LOYALTY AND DEFECTION
User-facing systems easily access MODELS APPLIED
scores and strategies in real time.
Customer Data “Customerization”
Data Refinery
Warehouse
9

11. Fitting in to the Current Infrastructure…
 Social media is just another customer touch point.
 The text we mine from social media outlets is another set of data about the customer,
just like the call center, website, or the customer surveys.
 We’d like to use that data just like the rest of the customer data: to help us predict customer
purchase behavior.
 In some sense social media provides a source of unsolicited surveys.
Users & business interface Customer Data by Touch Point
Dealers Call Center Click-thru Mail Call Center
Data Creation
Sales Dealer Service
Customers Website
Comments /
Survey
Twitter, FB, … Sentiment
LOYALTY AND DEFECTION
User-facing systems easily access MODELS APPLIED
scores and strategies in real time.
Customer Data “Customerization”
Data Refinery
Warehouse
10

12. A Source of “Unsolicited Surveys”…
 Why do we survey customers? From the narrow perspective of someone who predicts
customer behavior, the graph below is a big reason why.
 What’s more compelling is that
relationship between a customer’s
opinion and loyalty holds up when
we control for predicted loyalty.
 The “Loyalty=1” group is the group
that scores in the lowest 20% of a
loyalty model…a model built
without survey data.
 The results remain consistent
within each loyalty tranche so
much so that customers within
group 5 can have lower repurchase
rates than those in group 3,
depending on survey response.
 How much do we spend on surveys?
 Whatever it is, our feeling is that if we can establish the above relationship with social media
sentiment (use it as your X-axis), and cover more customers for less than what we currently
spend on surveys, then we have the beginning of a business case for extracting sentiment
from social media.
11

13. A Quick Digression on Business Cases…
 We believe that there are three “pillars” for the business case to
actively engage consumers through social media outlets (specifically
Twitter):
1. Conquesting new customers
2. Concern resolution
3. Voice of Customer
STRATEGY: Introduce Social Media as an additional
consumer touch-point
SUPPORTS THE 1. Conquest 2. Concern Res 3. VOC
STRATEGY: $XX mils per year $XX mils per year $XX mils per year
12

14. How We Use Survey Data In the Models…
 Let P = probability a Ford customer will repurchase another Ford upon disposing of
any one of his current Fords.
 Logistic regression is a very popular way to model and predict P.
 ln[p / (1-p)] = b0 + b1*x1 + b2*x2 + … bn*xn
 b0, b1, b2 are parameter estimates. They quantify the extent to which x1, x2, …, xn affect
the probability of repurchase.
 x1, x2, …, xn are explanatory variables, e.g. # of previous Ford purchase, time since most
recent Ford purchase, miles from nearest Ford dealership, etc…
 Now let s1 = 1 if “very likely”, 0 otherwise
 Let s2 = 1 if “likely”, 0 otherwise …
 Let s5 = 1 if “not at all very unlikely”, 0 otherwise
 Refit the logistic regression:
 ln[p / (1-p)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5
Can be thought of as the VOC (Voice of Customer) Index, but it’s based on just
survey data, which may only be available on 10% (roughly) of the customers.
This is a nice metric, because it, by design, predicts loyalty.
13

15. How We Use Twitter Data In the Models…
 Let’s treat the Ford customer’s “tweets” the same way we treat survey data.
 Go back to our logistic regression:
 ln[p / (1-p)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5
 And let t1 = 1 if we can identify a “Ford positive” tweet for the customer, 0
otherwise.
 Let t2 = 1 if we can identify a “Ford neutral” tweet for the customer, 0 otherwise.
 Let t3 = 1 if we can identify a “Ford negative” tweet for the customer, 0 otherwise.
 Refit the model:
 ln[p / (1-p)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5
+ bn+6*t1 + bn+7*t2 + bn+8*t3
Can be thought of as the BUZZ INDEX, and it comes directly from what
ford customers are saying on Twitter. This metric also, by design,
predicts loyalty. So this is a quantification of the BUZZ EFFECT.
14

16. Interpreting the “Twitter-Enhanced” Model…
 ln[p / (1-p)] = b0 + b1*x1 + … bn*xn + bn+1*s1 + bn+2*s2 + bn+3*s3 + bn+4*s4 + bn+5*s5
+ bn+6*t1 + bn+7*t2 + bn+8*t3
 When we fit this model, we get an intuitive result:
 bn+6 > bn+7 > bn+8 => good tweets lead to higher loyalty than do neutral tweets,
neutral tweets lead to higher loyalty than bad tweets.
 Not as intuitive (but interesting nonetheless):
 All three parameters are greater than 0 (implying ANY tweeting is better than
no tweeting).
 bn+8 (the parameter for bad tweets) is NOT SIGNIFICANT. There is not a
sufficient volume of bad tweets to support a significant result. The large
majority of FLM tweets are good.
 We think that the upshot of all of this is that tweeting about Ford, irrespective of
sentiment, signifies a high level of customer engagement. This has implications
beyond our efforts to better predict loyalty.
15

17. The Buzz Variables Improve the
Loyalty Model: So What?
 More data and better data yield models that do a better job of “separating” loyalists
and non-loyalists.
 One way this manifests itself: ranking the population of customers with a better model
will yield higher repurchase rates in the top decile (or demi-deciles…depending on how
many groups you want to establish), and lower repurchase rates in the bottom decile.
 So a marketing campaign that increases everyone’s likelihood of repurchase by 15% (not
an uncommon number), does so on a larger base of loyalists within the top decile, and
thus creates more incremental sales for the same amount of mailings.
 Say the difference between these two bars is 200 bps.
 Some of the incremental sales from the campaign
Repurchase Rate
Old Model
noted in the bullet above (top decile only), are
Model w/VOC & Buzz attributable to having a better model.
 How many? .02 * .15 * top decile population
 If the population of interest is 250,000
customers, then the impact of the better model is 750
incremental repurchases.
 If a repurchase is worth $5,000 profit, then the case
1 2 3 4 5 6 7 8 9 10 for the “buzz variables” is substantial: $3.75 million.
Model Decile
Low Loyalty to High
16

18. Why We Will Regularly Re-Fit the Buzz Index
 We have several reasons to believe that results may change when we re-fit our
enhanced loyalty model (twitter sentiment data being the enhancement):
1. The number of Tweeters is increasing all the time. Ford’s customer email capture isn’t
great but it is improving, and there is evidence that Ford customers are, relatively
speaking, very active on Twitter.
2. Attribution of tweets to customers is difficult and unsure; finding the Twitter names of
Ford customers is difficult and painstaking.
3. Classifying the sentiment of Tweets is an imprecise exercise, especially when using off-the-
shelf tools and software.
4. The content of the “Ford-tweeting” population leads to potentially biased results; it is
biased toward a demographic that naturally tends to be less Ford loyal:
 Females
 Young
 Used vehicle owners
 Appear to be more service loyal, which is a good thing
5. The tweeting population also happens to be geographically biased, but this does not
concern us as much as #4.
17

19. Re-Fit the Buzz Index:
The Increasing Number of Tweeters
 According to GIGAOM (http://gigaom.com/), Twitter had 175 million users in December 2010, and
was growing by 370,000 new users every day. Also as of 12/2010:
 65% of those users lived outside the US.
 Roughly 6% of all Americans were active on Twitter. More recent studies indicate the number is 9%-10% (or
13% of internet users).
 Of the Ford customers active as of 12/2010 (who had a valid email), we were able to find 9% of
them active on Twitter.
 Not surprisingly we have found the number of Ford related tweets to be increasing over time.
 Good and neutral tweets have increased whereas bad tweets have stayed flat. Seems like a good thing, but
we have some thoughts on comment classification.
Frequency of Ford-Related Comments Found on Twitter Ford-Related Twitter Comments "Binned"
60,000
6000 60000
50,000 Negative
5000 50000
Positive
40,000
Postive, Negative
4000 Neutral 40000
Neutral
30,000
3000 30000
20,000 2000 20000
10,000 1000 10000
0 0 0
Aug-08
Apr-09
Aug-09
Apr-10
Aug-10
Jun-08
Sep-08
Jun-09
Jun-10
Oct-08
Nov-08
Dec-08
Feb-09
Sep-09
Oct-09
Feb-11
Nov-09
Dec-09
Feb-10
Sep-10
Oct-10
Nov-10
Dec-10
Sep-08
Sep-09
Sep-10
Nov-08
Nov-09
Nov-10
May-08
Jul-08
Jan-09
Mar-09
May-09
Jul-09
Jan-10
Mar-10
May-10
Jul-10
Jan-11
Mar-11
May-08
Jul-08
Jan-09
Mar-09
May-09
Jul-09
Jan-10
Mar-10
May-10
Jul-10
Jan-11
Mar-11
18

20. Re-Fit the Buzz Index:
Attributing the Tweets
 Twitter names can be found if you can supply an email.
 With an email address you can find, through the Twitter API, a Twitter name, a first
name, and a last name associated with that email address. It will not return the email.
 Ford has 11 million bought-new still-retained customers. We have emails on several
million of them.
 We cannot run several million emails through the Twitter API. Even if they could be
processed, we would not be able to get back the email. We would only get back the
thousands of Twitter names associated with, but not matched to the emails.
 The only sure way to attribute emails to Twitter names is to go through the API one
email at a time. Can we off-shore this? We can, but we still will not be done until 2013
if we go this route. So we had to be more clever…
 Whatever method we choose, we need to recognize the Twitter name as useful
customer data, and as such, store it in the data warehouse. Next slide…
19

21. Re-Fit the Buzz Index:
Attributing the Tweets – You Must Retain Data
 What comes out of the attribution process is a table that looks like this:
EMAIL TWITTER_NAME FIRST_NAME LAST_NAME
kshields@marketingassociates.com Keith Shields
rleibovitch@marketingassociates.com ronedog Roni Leibovitch
mdeatrick@marketingassociates.com Mindy Deatrick
 We have another process (using RADIAN6) that scours Twitter for comments
that contain words in our “start list” (e.g.
Ford, Lincoln, Mercury, Taurus, Mustang, Fusion, etc…). It produces a table
that looks like this:
TWITTER_NAME COMMENT SENTIMENT
ronedog My new Ford Focus is also imported from Detroit. Good
20

22. Re-Fit the Buzz Index:
Attributing the Tweets – You Must Retain Data
 We have to integrate this into existing data warehouse processes (which
should be easy enough, if we’re treating this like just another source of
customer data):
Customer Touchpoint
EMAIL TWITTER_NAME FIRST_NAME LAST_NAME
Twitter Data Creation kshields@marketingassociates.com Keith Shields
rleibovitch@marketingassociates.com ronedog Roni Leibovitch
mdeatrick@marketingassociates.com Mindy Deatrick
Customer Data
TWITTER_NAME COMMENT SENTIMENT
Warehouse ronedog My new Ford Focus is also imported from Detroit. Good
21

23. Re-Fit the Buzz Index:
Classifying the Tweets…
 Prepackaged comment binning algorithms are not as accurate as we’d like…they result
in a high instance of inappropriate comment binning (“Positive”, “Neutral”, or
“Negative”). Here are some actual examples of inappropriately binned comments:
 Positive: “Classic Car For Sale 2001 FORD EXPLORER - Mt. Royal NJ: Runs and Looks Great!!!”
 Negative: “You have insulted my Ford Fiesta, shame on you.” AND “Just drove a Ford Fiesta
getting 30 mph. Not bad!” AND “Just dropped my car off at the FORD Dealership . I want a
FORD Fusion soooooo BAD.”
 Neutral: “Ford Escape!!!” AND “Smart, easy, & fun ride with the new Ford Focus with Ford
Sync's help!”
 In order for us to get the intuitive results we showed on slide 12 we had to depart from
sentiment classification algorithms and do a “brute-force” classification.
 Interns, Cornerstone Schools, Detroit, Mi. (http://www.cornerstoneschools.org/)
 There are other inexpensive ways…all of which we believe to be more accurate than existing
“machine intelligence”…albeit not as scalable:
• Existing call center personnel
• Mechanical Turks (https://www.mturk.com/mturk/welcome)
22

24. Re-Fit the Buzz Index:
The Biased, But Changing, Population of Tweeters…
 See the graph on the right. The fastest growing population of tweeters is 18-34 year
olds. About 56% of tweeters are 34 years or younger.
 The challenge for Ford: the median age for Ford customers is well above 34, despite
some recent strong entries in the small car market.
 The most “tweeted about” Ford is, not surprisingly, the Focus.
23

25. Re-Fit the Buzz Index:
Geographically-Biased Population of Tweeters…
 The numbers represent how
much higher the Twitter use
per capita is in that state
versus the nation as a whole.
 For example: if the national
usage rate is 10%, then
Michigan is 11% lower than
that: .10 - .11(.10) = 8.9%.
24

26. Given the Worries About Attribution, Classification…
 The real opportunity could lie in the business experts using intuition and
common sense to tailor campaigns and programs to tweeters based on
their most recent comments.
 This would only rely upon a good mechanism for scraping relative comments
from Twitter and reacting procedurally and appropriately.
 If we look at the comments as unsolicited survey responses we see
opportunities for customized offers and programs (no models needed –
the comment reveals the customer’s intent):
 Private Sales Offer and/or Pre-approval: “Just dropped my car off at the FORD
Dealership. I want a FORD Fusion soooooo BAD.”
 Rewards Program Offer: “My moms taking me to get this Ford explorer in the
mornin tho i should have a new whip before July then im haulin ass to the A”
 Offer for trade-in: “Ford focus sucks. Very uncomfortable vehicle.”
 Offer for service discount / extended warranty: “My car is running rough and
keeps blowing the injector and on plug coil fuses, its a 2006 3.0 V6 Ford
Fusion. HELP!!”
25

27. So We Revisit Our Three Pillars…
 We believe that there are three “pillars” for the business case to
actively engage consumers through social media outlets (specifically
Twitter):
1. Conquesting new customers: pay attention to influencers
2. Concern resolution
3. Voice of Customer
STRATEGY: Introduce Social Media as an additional
consumer touch-point
SUPPORTS THE 1. Conquest 2. Concern Res 3. VOC
STRATEGY: $XX mils per year $XX mils per year $XX mils per year
26

28. Conquest New Customers: Influencers
Fact: 93.6% of Twitter users have less than 100 followers, while 98% of users have less than
400 followers. Meanwhile, 1.35% of users have more 500 followers, and only 0.68% of
more than 1,000 followers.
27

29. Conquest New Customers: Influencers
Fact: As Twitter users attract more
followers, they tend to Tweet more often.
This is particularly evident once someone has
1,000 followers the average number of
Tweets/day climb from three to six. When
someone has more than 1,750 followers, the
number of Tweets/day rises to 10.
Fact: A small group of Twitter users account for the bulk of activity. Sysomos discovered that 5% of users
account for 75% of all activity, 10% account for 86% of activity, and the top 30 account for 97.4% of
activity.
28

30. Conquest New Customers: The Opportunity
 Find out who is expressing in-market sentiment and send them a targeted offer.
 We estimate that through Twitter alone, roughly 35,000 customers per year express
inclination to buy Ford.
 Applying a result from an analysis of "handraiser campaigns", we assume 15% of the
35,000 will purchase FLM. This is 35,000 * 15% = 5,250 sales.
 Assuming 20% lift from a targeted offer to in-market customers (not an uncommon
number), we estimate that a conquesting campaign directed at in-market "social-media
leads“. This is 5,250 * .2 = 1,050 incremental sales.
 In 1Q2012 Ford will run a program that tests the opportunity we have framed in the
above bullets. We will have the capability to offer differentiated treatment to influencers.
Marketing Associates built the process to find the tweets and measure the back-end
results.
 INTEGRATION will be through the customer data warehouse and EXECUTION through the
concern resolution center.
 At this point we won’t trouble you with another infrastructure diagram.
29

31. Some recommendations…
 Treat social media as another source of relevant customer data.
 Comments about your product are, in some sense, unsolicited surveys. They can, like surveys
do, improve your ability to predict the behavior of your own customers.
 Pay careful attention to the integration of social media data. Integration
requires “customerization”, so subsequent customer behavior can be tracked.
 Attributing comments to customers is tricky. It can also be painstaking. The good news is that it
can be done cheaply.
 Correct classification of comments is essential to understanding the true signal
in the comments.
 The most accurate means of classification may also be the least scientific: have an English
speaker (who preferable understands colloquialisms) read the comments, and bin them.
 Categories can be “good”, “bad”, “in-market”, “service issue”, or whatever aligns with the
differentiated treatments and offers.
 Retain data, and integrate intelligently into a data warehouse. Test and
measure several tactical approaches to customers and prospects who are
commenting about your products.
30

32. Quantifying the "Buzz" Effect: Integrating Social
Media with Loyalty & Defection Models
Questions?
Thanks for your time and attention.
30

33. APPENDIX
 From www.dealer.com:
 58% of vehicle purchasers posted a comment or status update on Facebook about their new
vehicle.
 41% of those who used social media while shopping, said they saw a post that caused them to
add a brand or model to their consideration. 28% saw a posting that caused them to add a
dealership to their consideration.
 74% of those who use Facebook said they would likely "Like" a manufacturer if they were offered
exclusive offers and savings.
 Methodology:
Approximately 2,000 recent vehicle purchasers and in-market shoppers were surveyed about
their use of social media. The study was based on McKinsey Consulting's consumer decision
journey model. The research was conducted by GfK Automotive.
30