The ROI of Engagement in Social Media
A presentation to SOBCon
Chicago, IL
May 2009
Katie Delahaye Paine
CEO
KDPaine & Partners, LLC
Durham & Berlin, NH
kdpaine.blogs.com
www.kdpaine.com
People talk. We listen.

Who I am
Data-driven
Insightful
Transparent
Full of ideas
Connector
Passionate about measurement,
gardening and politics
CEO of a NH-based 20-person research
firm specializing in measuring
relationships, engagementlisten. the impact
and
People talk. We

Conquering your fears
I’m afraid that metrics • If it’s not working, why keep
will reveal that my doing it?
program isn’t working
I’m afraid of what I’ll • If you’re deaf to thewill hear it
conversation,
only your enemies
hear
I’m afraid I won’t be • It’s not about justifying, it’s about
able to justify my improving
program/existence
I’m afraid I’ll be fired • You should be fired for not
for not showing the showing any numbers
right numbers
3
People talk. We listen.

Communications then and now
21st Century Role of
Traditional role of
Marketing & PR
Communications
4
People talk. We listen.

A measurement timeline
Social
MSM Online
Media
Eyeball
HITS Engagement
counting
Page 5
People talk. We listen.

The changing “Holy Grail” of measurement
6
People talk. We listen.

The measurement fork in the road
Improving
Selling stuff Reputation/relationsh
ips
To fix this Or get to this
7
People talk. We listen.

Goals drive metrics, metrics drive results
Reputation
/
Sales
Goal Relationshi
ps
Relation Engage
ship ment
scores Index
Recom Web
Metri mendati analytic
ons s
cs Positioni
Sales
ng
Marketin
Engage g Mix
ment Modelin
g
8
People talk. We listen.

What do you need to measure?
Outputs?
Did you get the coverage you wanted?
Did you produce the promised materials on time and on
budget?
Outtakes?
Did your target audience see the messages?
Did they believe the messages?
Outcomes?
Did audience behavior change?
Did the right people show up?
Did your relationship change?
Did sales increase?
9
People talk. We listen.

A Proposed Engagement Index
Output Outtake Outcome
Time on site Clickthru
Relationships
Repeat visits Donations/orders
Tone/content of
Forwards/links Signups
conversation
/comments
+ +
Membership
One Number?
Page 10
People talk. We listen.

Goals, Actions and Metrics
Goal Action Output Metric Outtake Outcome Metric
Metric
Increased Twitter Number of % inclined to % increase in online
in campaign tweets support donations
membersh % % increase in
ip improvement memberships
in perception
of organization
Increase in Flickr photo Number of % likely to Total online
donations contest entrants support the donations
cause
Message Blogger/Spe% % hearing % consistency in
consisten aker training postings/ar message messaging
cy ticles between external
% believing
containing message and internal
one or communications
more key Market share
messages Ratio of on-
11
People talk. We listen.
Share of message to off-

The 7 steps to Social Media
1. Define the “R” – Define the expected
results?
Define the “I” -- What’s the
2.
investment?
Understand your audiences and what
3.
motivates them
Define the metrics (what you want to
4.
become)
Determine what you are benchmarking
5.
against
People talk. We listen.

Step 1: Define the “R”
What return is expected?
What were you hired to do?
If you are celebrating complete 100% success
a year from now, what is different about the
organization?
If your department was eliminated, what would
be different?
13
People talk. We listen.

Step 2: Define the “I”
What is the investment?
Personnel
Agency compensation
Senior Staff time
Opportunity cost
14
People talk. We listen.

Step 3: Define your audiences and how you
impact them
You audience is never “anyone with a pulse”
There are multiple constituencies
List every stakeholder
Where do they go for information?
What’s important to them?
What is the benefit of having a good relationship with
that stakeholder group?
Understand your role in getting the audience to
do what you want it to do
Raise awareness
Increase preference
Increase engagement
15
People talk. We listen.

Step 4: Define your Key Performance Indicators
(KPIs)
Cost savings Trust:
Efficiency Improvement in relationship
Cost per message /reputation scores with
communicated customers and
Cost per new lead/customer communities
acquired (Loyalty/Retention)
Productivity: Thought leadership:
Increase in employee Share of quotes
engagement/morale Share of opportunities
Lower turnover/recruitment Message penetration
costs
Positioning on key issues
Engagement: Improvement in
Ratio of posts to comments favorable/unfavorable ratio
% of repeat visitors Improvement in Optimal
16
People talk. We listen. Score (OCS)
% of 5+min visitors Content

What makes a perfect communications
KPI?
Gets you where you want
to go (achieves corporate
goals)
Is actionable by
individuals as well as
departments
Continuously improves
your processes
Is there when you need it
People talk. We listen.

Step 5: Define your benchmarks
Emerging benchmarks
Engaged = 3-13 comments per post
Hyper-engaged = 15-35 comments per post
After 3 days most comments are done, 14 days
max
Social Bookmarking momentum = 1 submitted item
every other day
Message should be communicated in 2 out of 5
blogs
Past Performance, pre/post action
Think 3
Peer
Underdog nipping at your heels
Stretch goal 18
People talk. We listen.

Step 5: Conduct research (if necessary)
First: find out what already exists
Web analytics
Customer Satisfaction data
Customer loyalty data
Second: Decide what research is needed
to give you the information you need:
19
People talk. We listen.

Step5: Selecting a measurement tool based
on your KPIs
Objective Metric Tool
Increase inquiries, web traffic, % increase in traffic Clicktrax, Web trends, WebSide
recruitment Story
#s of clickthrus or downloads
Increase awareness/preference % of audience preferring your Survey Monkey, Zoomerang,
brand to the competition
Engage marketplace Conversation index greater Type pad, Technorati
than .8
Rankings
Communicate messages % of articles containing key Media content analysis –
messages Dashboards
Total opportunities to see key
messages
Cost per opportunity to see
key messages
% aware of or believing in key Survey Monkey,
message Zoomerang,Vizu
20
People talk. We listen.

Your tool box needs:
1. A content source:
Google News/Google
Blogs
Technorati, Ice Rocket
Cyberalert, CustomScoop,
e-Watch
Radian 6, Techrigy
RSS feeds
Twitter Search
eNR, Meltwater
Survey
Monkey/Zoomerang We listen. 21
People talk.

Your tool box also needs to include:
2. A way to analyze
that content
Automated vs.
Manual Tools:
Census vs random •Woopra
•Hubspot Grader
sample
•Xinureturns
The 80/20 rule – •Twinfluence
Measure what •SPSS
•Excel
matters because
•Crimson Hexagon
20% of the content •www.tealium.com
influences 80% of the We listen. 22
People talk.

Standard classifications of discussion
• Responding to criticism
• Acknowledging receipt of
• Giving a shout-out
information
• Making a joke
• Advertising something
• Making a suggestion
• Answering a question
• Making an observation
• Asking a question
• Offering a greeting
• Augmenting a previous post
• Offering an opinion
• Calling for action
• Putting out a wanted ad
• Disclosing personal
• Rallying support
information
•
• Recruiting people
Distributing media
•
• Showing dismay
Expressing agreement
•
• Soliciting comments
Expressing criticism
•
• Soliciting help
Expressing support
•
• Starting a poll
Expressing surprise
•
• Validating a position
Giving a heads up
People talk. We listen.

Standard classifications of videos
Advertisement Montage
Animation Music Video
Demonstration News Broadcast
Event/Performance Promotional Video
Fiction Sightseeing/Tour
Film Slideshow
Home Video Speech
Instructional Video Television Show
Interview Video Log
Lecture
People talk. We listen.

Your tool box also needs to include:
3. A way to measure engagement
The conversation index=
• Ratio of posts to comments
Relationship studies
The engagement index
25
People talk. We listen.

Share of conversation vs share of engagement
Share of Engagement by Subject - ,External Blogs
Share of Subject
Students 23.6% 33.2% 22.1% 21.1%
Research, Social Sciences 1 4 1
Staff 100.0%
Campus Life 1
Research, Social Sciences 4.4% 95.6%
Institution, Overall 2 1 3 Research, Physical Sciences 38.3% 2.3% 31.0% 28.4%
Research, Other
Policies 2
Research, Life Sciences 13.0% 20.8% 13.0% 53.2%
Research, Agriculture 4
Research, Earth Sciences 86.8% 13.2%
Other 1
Research, Agriculture 100.0%
Legal News Projects, Non -Research
1 2 28.6% 28.6% 28.6% 14.2%
Policies 100.0%
Admissions 1 1 Peer 1
Peer 1
Partnerships
Staff Michigan State
Michigan State
1
Other
Peer 2
Peer 2
Research, Life Sciences 1 1 2 1 3
Legal News 43.3% 56.7%
Peer 3
Peer 3
Alumni Topics Inventions
1 1
Peer 4
Peer 4 Institution, Overall 5.8% 94.2%
Financials 2 1 2
Financials 68.7% 12.5% 18.8%
Projects, Non -Research 1 1 1 2
Faculty 15.3% 34.9% 6.3% 43.5%
Research, Earth Sciences 1 2 2 Events
Courses
Courses 28.6% 71.4%
1 2
Community Relations
Research, Physical Sciences 3 2 4 6
Campus Life
Students 5 2 1 7
Alumni Topics 96.8% 3.2%
Faculty Admissions
2 6 2 2 6 33.3% 66.7%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0 2 4 6 8 10 12 14 16 18 20
Page 26
People talk. We listen.

The vast majority of discussion in external blogs is
neutral.
Share of Engagement by Tone - External Blogs
Share of Tone
100%
30
29
94%
90%
25
80% 83%
70% 71%
20
20
60%
58%
58%
Negative
Negative
50%
15
Neutral
Neutral
14
Positive
Positive
40% 42% 42%
12
10
30%
29%
8
20%
5
5
14%
4 4 10%
3
2 6% 3%
1
0%
0
Arizona State Michigan State Penn State Purdue University University of Michigan
University of Michigan Purdue University Penn State Michigan State Arizona State
Page 27
People talk. We listen.

For all institutions, most postings were simply
making an observation or distributing media.
Share of Conversation Types Share of Engagement by Conversation Type - Institutional Blogs
Showing dismay
Showing dismay 100.0%
1
Recruiting people
Recruiting people 3 1
Rallying support
Rallying support 1
Playing a game
Playing a game 16 1
Offering an opinion
Offering an opinion 49.5% 10.8% 39.7%
2 11 2 4 1
Making an observation
Making an observation 30.9% 23.1% 10.9% 35.1%
14 46 6 18 9
Making a suggestion
Making a suggestion 72.7% 27.3%
15 12 3 6 8
Giving a shout-out
Giving a shout-out 5 12 2 2
Giving a heads-up 6.5%
Giving a heads-up 26.9% 66.6%
Arizona State
29 203 4 13 17
Arizona State
Michigan State Expressing surprise
Expressing surprise 1 Michigan State
Expressing support
Expressing support Penn State Penn State
1 2 2 3
Purdue UniversityExpressing criticism Purdue University
100.0%
Expressing criticism 3 1
cx
University of Michigan
Distributing media
University of Michigan 53.9% 46.1%
Distributing media 36 787 235
Disclosing personal information 44.2% 1.6% 38.7% 15.5%
Disclosing personal information 7 24 4 13 5
Calling for action
Calling for action 1 2
Augmenting a previous post 100.0%
Augmenting a previous post 6 1
Asking a question 100.0%
Asking a question 1 2
Answering a question
Answering a question 6 7 1
Advertising Something
Advertising Something 3 12 2 11
Acknowledging receipt of information
Acknowledging receipt of information 2
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Page 28
People talk. We listen.

Aspects of relationships
Control mutuality
Trust
Satisfaction
Commitment
Exchange relationship
Communal relationship
29
People talk. We listen.

Components of a Relationship Index
Control mutuality
In dealing with people like me, this organization has a tendency to throw
its weight around. (Reversed)
This organization really listens to what people like me have to say.
Trust
This organization can be relied on to keep its promises.
This organization has the ability to accomplish what it says it will do.
Satisfaction
Generally speaking, I am pleased with the relationship this organization
has established with people like me.
Most people enjoy dealing with this organization.
Commitment
There is a long-lasting bond between this organization and people like
me.
Compared to other organizations, I value my relationship with this
organization more
Exchange relationship
Even though people like me have had a relationship with this
organization for a long time; it still expects something in return whenever
it offers us a favor.
This organization will compromise with people like me when it knows that
it will gain something.
This organization takes care of people who listen.
People talk. We are likely to reward the 30
organization.

How to implement relationship metrics
Step 1: Define your universe
Step 2: Sign up for Survey
Monkey/Zoomerang
Step 3: Conduct a benchmark
relationship study
Step 2: Implement your program
Step 3: Conduct a follow up relationship
study
Step 4: Look at how your numbers have
changed People talk. We listen.

Metrics that show engagement with your
own site
% increase or decrease in unique visits
Change in page rank - i.e a list of the top ten most
popular areas and how it has changed in the last
week
How many sessions on our blog or web
site represent more than 5 page views
In the past month, what % of all sessions
represent more than 5 page views
% of sessions that are greater than 5 minutes in
duration
% of visitors that come back for more than 5
sessions
% of sessions that arrive at your site from a Google
search, or a direct link from your web site or other
site that is related to your brand
Courtesy %Eric Peterson that become a subscriber
of of visitors
% of visitors that download something from the site
Page 32
% of visitors that provide talk. email address
People an We listen.

Metrics that show engagement with
other sites
Conversation Index
Links
Forwards
Digg, Stumbleupon
Facebook, Twitter, Pownce, Utterz, Seesmic
Comments
Ratings
Ranking
Time between posts
Tone of discussion
Page 33
People talk. We listen.

Your tool box needs to include:
4. A way to quantify it all
HITS= How Idiots Track
Success
Eyeballs – Compete
Google Analytics
Panels
Surveys
34
People talk. We listen.

Step 7: Analysis
Research without insight is just trivia
What works, what doesn’t?
What needs to be done?
What are you communicating?
What tools work best?
35
People talk. We listen.

Data mining the numbers you have
Look for failures first
Check to see what the competition is
doing
Then look for exceptional success
Compare to last month, last quarter, last
year
Figure out what worked and what didn’t
work
People talk. We listen.

Best Practices:
Correlations to bottom-line Benchmarking against
impact your peers
Donations Looking at what the best do
Memberships Setting goals accordingly
Sign-ups Use data to persuade
recalcitrant spokespeople
Leads
Social Media in Crisis
Using SMM for planning
Listen instantly to a wide
Define the time frame,
range of influencers
market/topic you want to
study Identify weaknesses in
communications, customer
Use Google News,
service, or in the product
Technorati or Radian6 to
identify the conversations Improve your reputation
around the topic Listen first, then respond
Analyze the conversations Stop doing stupid things
People talk. We listen.
for type, tone and positioning

Changing reputation via metrics
Tone of Conversation over time
60
50
30
40
2
30 16
Positive
Mentions Neutral
17
Negative
5
20
12
27
4 24
2 1 2 20
3 9
10 16 15
8 8
10
5 9 9
4
2 6 5
4 4 4 4
2 2 2 2 2 2 2
0
Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec
2007 2008
People talk. We listen.

Negative coverage over time
25
1
3 1
20 2
2
4
15
1
Entries
14
21 15
18
10
10 14
5
1
10 2 12 10 2
9
5
3
4 2 4 2
7
6
5 1
4 4 2 2 2
2 2 2 2 2 2 2 2
1 1 1 1 1 1
0
Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr
2006 2007 2008
People talk. We listen.

Correlation exists between traffic to the ASPCA web
site and the organization’s overall media exposure
350,000,000 700,000
300,000,000 600,000
250,000,000 500,000
Web Site Visitors
200,000,000 400,000
Exposure
Overall Exposure
150,000,000 300,000
Web Traffic
100,000,000 200,000
50,000,000 100,000
0 -
People talk. We listen.

Tying activity to development/marketing
goals
350,000,000 $1,800,000
$1,600,000
300,000,000
$1,400,000
250,000,000
$1,200,000
Donations
Exposure
200,000,000 $1,000,000
Overall exposure
Online donations
$800,000
150,000,000
$600,000
100,000,000
$400,000
50,000,000
$200,000
0 $0
em e r
M ry
ne
br ry
ch
S Aug ly
r
Ml
cr
D em r
te st
ay
i
be
O be
ov e
pr
Ju
ec b
ua
Fe ua
ep u
N to b
Ju
ar
A
m
n
Ja
41
People talk. We listen.

Case Study: Engagement vs mentions
Users were positively engaged with
advertisements
March 2009 Share of Tone by Company Share of Engagement by Tone for March 2009
Negative Neutral Positive Negative Neutral Positive
100%
100%
90% 90%
80% 80%
70% 70%
60% 60%
50% 50%
40% 40%
30% 30%
20% 20%
10% 10%
0% 0%
Client Competitor 1 Client Competitor 1
Georgia-Pacific Kimberly-Clark Weyerhaeuser Georgia-Pacific Kimberly-Clark
Weyerhaeuser
Competitor 2 Competitor 2
People talk. We listen.

By percentage, individuals were more engaged with
Client subjects than competitors
(Engagement is the average number of comments per post made to a blog)
March 2009 Discussion by Subject March 2009 Share of Engagement by Subject
Georgia-Pacific Kimberly-Clark Weyerhaeuser Georgia-Pacific Kimberly-Clark Weyerhaeuser
Client Competitor 1 Client Competitor 1
Competitor 2 Competitor 2
100% 100%
90% 90%
80% 80%
70% 70%
60% 60%
50% 50%
40% 40%
30%
30%
20%
20%
10%
10%
0%
0%
43
People talk. We listen.

Household product discussion jumped from discussion
of a Greenpeace report on toilet tissue
Discussion by Subject Over Time 
120
Away from Home Products
5
2
Building Products
100
Company Activities
2
2 55
80
64 Environmental
2
Mentions
Issues/Sustainability/Global
Warming
3
2 62 Household Products
45 53
60 27
2 8
5 Legal Issues
37 37
45 4
8
43
40 11 33 5
25 Management/Employees/Unions
6
24 5 40
29
22 12 11 25
4
20 Office Products
5 5
18 24
7 5

10
6 9 8 9
3
5 1
11 3 3
4
3
2 Packaging (Color Box)
3 8 7 7
7 6
5 5
4 4
4 2
0
Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
2008 2009
People talk. We listen.

Thank you
For more information on measurement, check
out our blog: kdpaine.blogs.com or
Subscribe for free to The Measurement
Standard, www.themeasurementstandard.com
To start developing your own dashboard or for a
copy of this presentation go to:
http://www.kdpaine.com
Or call me at 1-603-868-1550
Or give us your business card and we'll be
happy to send it to you
Page 45
People talk. We listen.