Early in 2011, our team began with the hypothesis that relationships with a brand can be found online and that finding these relationships would be critical to determining the value and ROI of social media investments to date. With many in the industry moving to qualitative analysis of social media mentions (tagging by hand), we began to explore this as well at Thornley Fallis. But there was a problem. We were spending a lot of time and money tagging people's views of our client's products or offerings with no ability to measure and determine whether the brand had built more or less relationships of value. We began to search for new tools. We quickly disregarded Klout, PeerIndex and other similar tools as means of finding relationships. These tools were better at aggregate or synthetic benchmarks of "influence," and our team was interested in finding relationships with a brand beyond the most "influential" web celebrities. This deck outlines our approach and findings and how we were able to find what we are calling the consistently engaged. These people are showcasing their potential relationship with a brand, yet are often being ignored as a result of the tools and approaches currently available.

1. Social Media Monitoring
Finding Relationships
A Pilot Study
July 4, 2011

2. Your brand is social.
From toilet paper
to luxury vehicles,
social plays a part
in the customer
experience of your
brand.

3. People have always
talked about brands
and experiences

4. Only now these views
and experiences are
being shared online

5. And subsequently...
Agencies and brands are
starting to take social media
seriously.

6. With just about
everyone offering
some form of
monitoring and
analysis.

7. We are starting to ask
better questions.
Should we be only watching for brand
and product mentions?
What issues matter to our brand?
What is the value of our social media
efforts?

8. But we are ignoring a
key question.
?

9. But we are ignoring a
key question.
Why are we
throwing away
all of the data?

10. Report 1
Typically, a baseline social media audit is
created based on an analysis of brand
names, competitors, and relevant issues.

11. Report 1 Report 2
The next report, for a different time period, is
created from new data. If comparisons to the
first report are made, they are aggregate or
trend-based.

12. Report 1 Report 2 Report 3
With each subsequent social media report,
the process remains the same.

13. We have lots of
pretty charts
and surface-level
analysis.

14. Pretty charts.
“More people spoke positively
about kittens this week.”
1500
1000
Positive
Negative
500
0
Week 1 Week 2 Week 3 Week 4

15. …and “analysis”.
“Here are our top kitten
influencers this month.”
Twitter name Kitten Tweets Klout Score
@justinbieber 618 90,194,000
@aplusk 6 12
@britneyspears 42 315
@oprah 9 120

16. But are all these kitten lovers
net new people every
month?
Is this just a one time spike
because of something in the
news?
Are we building relationships
with any of these people?

17. A Real-World Example
We decided to use Nikon (geo-
filtered to Canada) as a pilot
case.
We explored using both Radian6
and Sysomos MAP.
We settled on Sysomos MAP to
pull the data we needed due to a
feature that allowed us to pull an
individual’s raw tweets with no
search filters applied.

18. Some Charts
Most of the tools can give us
access to data and clues that lead
to insights, but the built-in charts
aren’t that helpful on their own.

19. Influence in Sysomos
Sysomos gives us a list of the top Twitter
influencers based on an algorithmic authority
score of 7/10 or higher, sorted by volume.

20. It falls apart…
With this chart, if we dive deeper and look at these users,
we discover nearly all are spam or of little value to the
brand.
Are these truly the most passionate and
influential Nikon brand advocates?

21. Influence in Radian6
Some tools, like Radian6,
integrate Klout or other similar
theoretical metrics of absolute
influence.
But our attempts to generate lists
of people who care about a
brand with these tools generates
primarily spammers and big
name web celebrities.

22. We need deeper insights.
With web analytics and email
marketing, we track unique and
repeat visitors.
And traditional CRM programs
track preferences, purchases, and
engagement over time.

23. Deeper insights in social.
Why not track the same things
on social media?
Who are our true brand
advocates and what can we learn
about them beyond their interest
in our brand?
Do the people who are
passionate about our brand have
anything in common?

24. What about Social CRM?
Some are looking at Social CRM
to solve this problem.
Most CRM tools were designed
with the assumption of a near-
perfect signal-to-noise ratio. To
be useful, there should be zero
noise in the system.
These systems fall apart when
you start throwing hundreds of
thousands of users and tweets
at them.
Social CRM
e.g. There was an estimated 501,618 “Nikon” tweets
in the past 6 months.

25. A new approach.
What if we started to wonder
about the people who are
consistently talking about our
brand or issue?
What could we learn if we
weren’t wiping the slate clean
every time we run a new search?

26. A new approach.
We believe these are people who
are passionate about your brand.
We call them the consistently
engaged.
They are NOT the most influential
Klout or PeerIndex users.
They are the people who have an
ongoing relationship with your
brand. Good, bad or otherwise.

27. Looking across periods.
Report 1 Report 2 Report 3
Simply put, we find the individuals who are
consistently talking about and engaging with
your brand across time periods.

28. Our hypothesis
The consistently engaged who
love your brand are:
• Findable
• Of value to the brand
(promotions, research, reach)
• Ignored by most brands

29. Looking across quarters.
We extracted all tweets over a
two-quarter period for Nikon and
looked at two lists:
1. Top Twitter users by
volume (4+ mentions)
2. Top Twitter users by
volume (4+ mentions) who
are also consistent across
multiple quarters

30. Finding the consistently engaged users
Volume Consistent
The list on the left includes @bdicroce @dorisdays88
people with the most volume @caltek79 @jedgar
@CFN007 @justinlee_
over the past six months, in
@cicychan @maryellenphotos
regards to the brand in @Digital_zin @msjconnolly
question. @dorisdays88 @neek247
@ginette4 @nortonphoto
@jedgar @jonah_lewis
The list on the right includes @JohnBiehler @quotetasticc
people who consistently @justinlee_ @RajaKalsi
engaged with or mentioned @jwsutts @RDslva
the brand a minimum of four @kinematicdigit @Redawna
@KingKabuz @samobeid
times in both quarters.
@Kosmatos @scottoakley
@Lalalalal8 @ShaniceAshley_
@maryellenphotos @stephaniefusco
@missemcee @vickiesbphoto
@missfish @wifewithknives
@msjconnoly @ZtheWayfarer
@neek247 @zeefred

31. Finding the consistently engaged users
Volume Consistent
The green boxes show the @bdicroce @dorisdays88
people consistent between @caltek79 @jedgar
@CFN007 @justinlee_
the two methods.
@cicychan @maryellenphotos
@Digital_zin @msjconnolly
If you are using the standard @dorisdays88 @neek247
@ginette4 @nortonphoto
method (left side) then you @jedgar @jonah_lewis
are ignoring up to 70% of the @JohnBiehler @quotetasticc
people consistently engaging @justinlee_ @RajaKalsi
with your brand. @jwsutts @RDslva
@kinematicdigit @Redawna
@KingKabuz @samobeid
We need to focus on the @Kosmatos @scottoakley
@Lalalalal8 @ShaniceAshley_
users who are consistent in
@maryellenphotos @stephaniefusco
order to find the true brand @missemcee @vickiesbphoto
advocates. @missfish @wifewithknives
@msjconnoly @ZtheWayfarer
@neek247 @zeefred

32. Rethinking Influence
The list of influencers most social
media tools give us aren’t very
meaningful as they tend to use
the standard method on the left
or just find the big name web
celebrities.
With our Nikon example, they
ignore a huge segment of people
who are passionate about Nikon
but will never appear on any top
list.

33. The Consistently Engaged
Consistent
Finding these people was just the @dorisdays88
@jedgar
first step. @justinlee_
@maryellenphotos
@msjconnolly
Now we need to understand: @neek247
@nortonphoto
1. Who are these people? @jonah_lewis
@quotetasticc
2. What are their other interests? @RajaKalsi
3. Their relationship to our brand @RDslva
@Redawna
4. Their potential value @samobeid
@scottoakley
@ShaniceAshley_
@stephaniefusco
@vickiesbphoto
@wifewithknives
@ZtheWayfarer
@zeefred

34. Understanding these people
All social media monitoring tools
start with a filter – a way to
cross-section massive amounts
of data (in this case a search for
“Nikon”).
Analyzing this data to
understand who these people are
is critically flawed. It will only
show us insights based on a
limited set of their conversations
– those that match our original
search terms.

35. It’s akin to doing an ethnography
study where we ignore all the
things the individual does or says
that don’t directly involve our
product. Not particularly valid.

36. So we remove the filter.
The “from:username” operator in
Sysomos MAP allows us to pull all
user tweets over any period of
time (sample based in some
cases).
This is time consuming to do but
not overly difficult. But now we
have hundreds of thousands of
lines of data.
How do we analyze this?

37. Not in the SM tool, sadly
The text analytics tools in Sysomos MAP quickly fall
apart when the filter is removed and we are looking
at everything a group of users say.

38. So we need to export all
the activity and analyze
it using a qualitative
tagging methodology

39. Qualitative vs. Quantitative Research
Quantitative content analysis is
about counting content. It
provides numbers that represent
frequencies or occurrences.
Qualitative content analysis is
about the quality of content. It
provides insights into recurring
themes that can lead to the
development of valuable insights
and recommendations.
Source: Zhang, Y. , & Wildemuth, B. M. (2009). Qualitative analysis of content. In B. Wildemuth (Ed.), Applications of
Social Research Methods to Questions in Information and Library

40. Qualitative vs. Quantitative Research
Rather than reporting how many
times a brand has been
mentioned [quantitative], we can
look at the quality of the
message [qualitative].
Rather than reporting
frequencies, the application of
qualitative tools to social media
data will allow us to gain insights
from a data sample that is
traditionally overlooked.

41. Conducting a
Conducting a
qualitative analysis
qualitative analysis
of isolated
of all expressions
individual
from identified
expressions of
users of interest.
interest.

42. Qualitative Analysis Approach
1. Identify your area of interest.
2. Identify goals: e.g., understanding what common characteristics
and interests of a specific customer population might exist.
3. Translate your goals into a structured and hierarchical coding
frame (either inductively or deductively).
4. Make both categories and individual codes as mutually exclusive
as possible.
5. Create coding rules to ensure that multiple coders can
consistently make the same coding decisions.
6. Test your coding frame by calculating inter-coder reliability (or
other fancy reliability method).
7. Assign expressions (or whatever your agreed upon unit of
analysis might be) to codes.
8. Analyse the coded data by individual category and code.
Compare codes, find relationships, uncover themes, discover
insights and develop recommendations.
Source: Zhang, Y. , & Wildemuth, B. M. (2009). Qualitative analysis of content. In B. Wildemuth (Ed.), Applications of
Social Research Methods to Questions in Information and Library

43. What did we find?
For this pilot, we analyzed the top
20 consistently engaged which were
responsible for over 20,000 social
media mentions in the quarters
analyzed.
Only a partial analysis was
performed to validate that the most
consistently engaged individuals
identified would be of value to the
brand.

44. What did we find?
Nearly all of our consistently engaged
users sample self-identify as amateur or
professional photographers.
Most have a relationship with Nikon –
either because they own Nikon
products or discuss Nikon products and
news.
1 out of 20 self-identified as a
professional photographer and 14 out of
20 stated they own Nikon equipment.

45. Their relationship with Nikon
Wants to buy a Canon
Recently bought a 60D but has an
Nikon D3100 and interest in Nikon gear
looking to purchase and often chats with
some additional lenses Nikon users
Professional Canon Angry with Nikon
shooter but blogged Canada about a poor
about a positive warranty service
experience shooting experience
with Nikon equipment
Owns a Nikon D300
and mostly uses older
manual-focus lenses. Hobbyist looking to
Just bought first move up to pro-level
“modern” Nikon lens. Nikon gear

46. Their other interests
Users are more likely to talk about iPhone vs. Blackberry
iPhones vs. BlackBerry devices
– 74% of all iPhone mentions were positive in
nature
– 73% of BlackBerry mentions were positive iPhone
in nature BlackBerry
iPhone users more likely to talk about
photo related apps than BlackBerry
users
– Almost half of all iPhone mentions had to
do with a photo-related app
– Only 4% of BlackBerry mentions were
about photo-related apps

47. Their other interests
The consistently engaged Nikon users Coffee Mentions
over the time period analyzed were
more likely to talk about Starbucks than
any other coffee brand
Starbucks
– 87.5% of these mentions were positive in
Tim Hortons
nature
– The second mentioned brand was Tim
Hortons, but all mentions were neutral or
negative in nature
People spoke of coffee 1.7 times more
often than tea.
Of the tea mentions, DavidsTea was the
most mentioned product/brand.

48. Quick Recap
1. The consistently engaged are
a better way to find the
relationships you are building
(or not building) in social.
2. Finding the consistently
engaged is simply a matter of
not throwing the data away
and paying attention to who is
engaging with your brand
across time periods.
3. Deeper analysis can lead to
better insights into who your
brand advocates are, their
needs and their interests.

49. The “Secret” Sauce

50. Our Methodology
A) Finding consistent users across two quarters
1. Extract two time periods of data from Sysomos MAP.
Export.
2. Merge data into a single spreadsheet with each
quarter in a separate column.
3. Cross-reference columns to find users who are
consistent across quarters.
4. De-dupe results in the third “result” column to get a
complete list of consistent users.

51. Our Methodology
B) Determining tweet volume for consistent users
1. Merge Tweet content from both quarters into a single
speadsheet column
2. Sort alphabetically and run subtotal based on
usernames. Sort subtotals by volume.
3. Set a threshold (we used 4 tweets across quarters)
4. Cross-reference with list (result of part A) of
consistent users across quarters.
5. Manually remove any spammy users above threshold.
6. Extract user list of top 20-30 “casually engaged.”

52. Our Methodology
C) Removing the filter and exporting all user tweets for
“casually engaged” list
1. Using the “from:username” operator in Sysomos MAP,
extract all top 20-30 user tweets to CSV over same
two quarter period (Export is limited to 5000 tweets
so generally each user must be done separately)
2. Merge resulting CSV files for each user into a single
spreadsheet. Discard all data except tweet content.
3. Set up qualitative tagging system and methodology.
4. Import data in NVivo or similar qualitative analysis
tool.

53. Our Methodology
D) Running qualitative analysis to find insights about
users
1. Identify an area of interest and goals. Translate these
into a structured coding frame.
2. Tag the data according to the coding frame.
3. Analyze the data based on the categories and codes.
Pull insights and findings.
4. Put together recommendations based on key findings

54. Colophon
Sean Howard is a freelancer and Eric Portelance is a Strategist at Katie Charbonneau is an
Associate at Thornley Fallis and Thornley Fallis and a regular Account Coordinator at
spends his life searching for what podcaster on the show Attention Thornley Fallis with an MSc in
drives and identifies the most Surplus. He’s passionate about using Media and Communications.
passionate online and offline. new technologies to build engaging She’s a data junkie with a keen
online experiences. interest in American politics.
Twitter: @passitalong Twitter: @eportelance