Presentation I and a colleague gave for the Brandwatch Masterclass in San Francisco.

1. NLP
+
Brandwatch Analytics
Deriving
insights
from
social
conversations using Natural
Language
Processing and
the
Brandwatch Analytics
API

2. What
we
will
cover
today
What
do
we
want
to
answer?
(and
why)
Our
approach
to
social
data
Leveraging
the
Brandwatch API to
extract
data
Deriving
insight
from
personas
Identifying
key topics
of
conversation
Segmenting
on
those
topics
to develop
personas
Replicating
back
into
Brandwatch
What
we’re
working
on
next

3. 3 |
§ Provides
an
in-­‐situ
portrait
based
on
exhibited
behavior
not
on
elicited
feedback
§ Highly
relevant
as
it
can
be
updated
in
near-­‐real
time
§ Enables
research
budget
to
be
focused
on
insights
rather
than
data
collection
Social
intelligence
enables
new
ways
of
answering
traditional
business
questions
and
driving
data
driven
actions
What
are
the
sort
of
questions
we
want
to
answer?
How
can
a
financial
services
company
reach
out
to
cyclists?
How
can
we
get
small
business
owners
to
engage
with
their
cell
phone
provider
online?
What
is
the
customer
journey
for
a
motorcycle
enthusiast?

4. 4 |
There
are
seven
stages
to
the
analytical
process
of
developing
utilizing
personas
with
social
data
Our
approach
to
working
with
social
data
Extract
Develop
the
dataset
Linguistic
model
Segment Analyze TrackQuery
data
Prepare
§ Need
to
truly
understand
your
data
before
any
analysis
§ Iterative
query/dataset
development
through
virtual
ethnography
§ Use
the
Brandwatch API
to
extract
the
full
text
mentions
Model
§ Employ
Natural
Language
processing
to
model
how
people
talk
§ Use
either
qualitative
methods
or
clustering
algorithms
to
segment
Understand
§ Through
visualization
and
analysis
we
can
understand
thoughts,
feelings
and
preferences
§ Replicate
back
into
Brandwatch
as
sub-­‐categories
to
monitor
on
an
ongoing
basis

5. 5 |
• Provide
the
basis
for
a
‘corpus’
in
NLP
jargon
from
which
to
model
• We
have
built
a
library
of
functions
using
python
to
retrieve
and
format
the
data
• The
output
format
of
the
API
is
in
JSON
so
there
is
some
work
to
turn
it
into
a
table
we
can
read
and
use
Extracted
BW
data
has
many
use
cases,
today
we
will
be
primarily
focused
on
full
text
mentions
Leveraging
the
API
to
extract
the
dataset
Example
API
function:
def get_mentions_query_URL( startdate,enddate,project_id,
query_id,access_token,fullText):
query_def = "data/mentions”
end_date = "endDate=" + end_date + "T00:00:00.000Z”
start_date = "startDate=" + start_date +
"T00:00:00.000Z"
request_URL="https://newapi.brandwatch.com/projects/"
+str(project_id) + "/" +query_def
if fullText == True:
request_URL = request_URL + "/fulltext"
request_URL = request_URL + "?" + "queryId=" +
str(query_id) + "&" + start_date + "&" + end_date +
"&pageSize=5000" + "&access_token=" + access_token
return request_URL
Read
more:
blog.tahzoo.com/tech-­‐thursday-­‐brandy-­‐py-­‐a-­‐python-­‐library-­‐for-­‐brandwatch/
Github:
https://github.com/BillmanH/brandy.py/

6. 6 |
Linguistic
model
-­‐ Identifying
the
topics
in
a
conversation
pumpkin
sugar
HEALTHY
LIVING
PUMPKIN
SPICE
CONVERSATIONS
TEXT
ANALYSIS
TOPIC
MODEL
1 Break
down
each
conversation
into
the
words
and
sentences
to
probabilistically
assess
each
word’s
relationship
with
each
other
word
2 Analyze
to
uncover
the
most
common
“topics”
of
conversation
3 Run
clustering
analysis
to
segment
on
topics
4 Iterate
on
topics
until
we
develop
a
solid
segmentation
Four
steps
to
targeting
personas

7. 7 |
("pumpkin
spice
latte")
NOT
("vue pack"
OR "value
pack"
OR
"how
to
make"
OR
"win
free"
OR "latte
cake" OR
"black
friday"
OR "pack
of"
OR "My
TL
right
now
iOS7
Hump
Day
iOS7"
OR
site:(twitter.com OR kdvr.com OR fox59.com
OR
news.google.com))
An
example:
who
discusses
Pumpkin
Spice
Lattes?
Our
query…
Excluded
because
of
irrelevant
recipes
Purposefully
broad
query
to
capture
full
range
of
conversations
Exclude
Twitter
as
it
would
overwhelm
the
results
and
we
couldn’t
export
full
text
mentions

8. 8 |
Do
it
Yourself
Starbucks Nutrition
Healthy
living
Style Urban
living restaurants PS
recipes
Amazing
treat
Pumpkin
Spice
ingredients
PS
Flavor
Coffee
at
home
people pumpkin grams squash fall city binary milk love food pumpkin home
make spice fat healthy wear place victoire pumpkin time hari spice inch
things latte calories recipe fashion food restaurants coffee day babe pie coffee
life starbucks sugar food boots park options sugar good ingredients latte green
time fall registers recipes style street time recipe back sugar flavor set
thing psl data copycat color local pst spice week science flavored keurig
feel drink saturated favorite wearing free visit cup great cancer seasonal count
find coffee carbs soup dress art trading make home organic year price
years today sodium paleo black event restaurant cream made found taste mountain
world lattes pos version top restaurant september syrup work chemical food make
The
topics

9. 9 |
DIY
Example:
“I
get
annoyed
when
a recipe
calls
for
pumpkin
pie
spice.
It's
not
that
people
use
it
that
annoys
me,
it's
the
mere
existence
of
it
as
a
single
spice.
…
I
guess
I'm
just
a
purist
at
heart.
Since
I
haven't
seen
pumpkin
pie
spice
here
in
France
I
now
need
to
make
our
own
pumpkin
pie
spice
mixture,
and
then
figure
out
the
right
proportions
for
my
dreamboat
pumpkin
spice
latte.
Nothing
that
a
Google
search
won't
solve,
but
annoying
nonetheless.
And
don't
worry,
when
I
do
I'll
be
sure
to
share
it
with
you.
Maybe
you'll
even
get
some
rainbows.
Fingers
crossed.
An
example
of
how
this
analysis
works
Treat/Reward
Routine
Example:
“I
thought
splurging
on
a
venti pumpkin
spice
latte
would
make
me
feel
better
this
morning,
(or
maybe
even
the
three
cups
of
green
tea
with
lotsa honey
in
it!)
...but
as
my
ears
pop,
my
nose
runs,
and
my
throat
feels
like
somebody
took
sandpaper
to
it
last
nite,
I
guess
it's
time
to
finally
suck
it
up
&
take
some
meds
ó¾Œ®ó¾ ‚
I
blame
you!
Rodney
Deal!!
Haha kidding
kidding
;
)
Below
are
two
pieces
of
verbatim
content
that
we
used
in
our
model.
The
first
post
is
connected
with
the
DIY
(62%
relevant)
topic
and
the
second
with
Treat/Reward
(73%
relevant)
62% 73%
DIY TREAT
/
REWARD
PS
FLAVOR FALL
(SEASON) PSL
RECIPES HEALTHY
LIVING
FILLER/
INFREQUENT
WORDS
TOPICS:

10. 10 |
• K-­‐means
highlights
clusters
of
conversations
based
on
the
topics
they
discuss
• This
creates
a
segmentation
that
reflects
how
people
discuss
a
subject
• Keys
in
on
the
pattern
of
topics
in
a
conversation
We
use
the
k
means
clustering
algorithm
to
segment
the
conversations
based
on
the
topics
in
order
to
create
the
personas
Segmenting
on
the
topics

11. 11 |
Urban
Living
Fall
(season) Dessert
Starbucks
drinks
Pumpkin
flavor
Treat
/
reward
Pumpkin
(recipes)
DIY
Fall
(season)
Treat
/
reward
Pumpkin
(flavor)
Dessert
StyleDesserts
Treat /
Reward
Being
Healthy
Urban
Living
Fall
(Season)
Starbucks
Drinks
Fall
(season)
DIY Desserts
Pumpkin
spice
recipes
Treat/
Reward
LESS
IMPORTANT
MORE
IMPORTANT
Grouping
the
topics
that
are
core
to
each
segment
we
can
see
where
differences
break
down
Mapping
topics
to
personas

12. 12 |
Plotting
continuums
to
understand
the
personas
Why
they
like
it
What
it
stands
for
NOVELTYNOSTALGIA
GUILTY PLEASURE
DAILY RITUAL
OPPORTUNISTICTRADITIONAL
PERENNIALSEASONAL
OFTEN
OCCASIONAL
EXPECTED
EARNED

13. 13 |
What
we
found
22%
PSL
PAMPERER A
pumpkin spice
latte
is
a
treat
to
be
savored
after
it’s
earned
or
after
a
tough
a
Monday
morning,
“What
a
weekend.
Hello,
slow
Monday.
Oh
what's
that?
I
should
get
a
pumpkin
spice
latte?
Well,
if
you
insist...”
34%
LATTE
CHEMIST
They make
their
own
lattes
in
the
comfort
of
their
own
home
or
tinker
with
the
official
version
“Here
is
an
awesome
home
version
of
Starbucks
Pumpkin
Spice
Latte.
Very
simple
to
make
and
alot
cheaper… personally
I
like
it
better
because
you
control
the
amounts
of
ingredients
you
put
in
it
according
to
your
taste.”
38%
FALL
FANATIC Pumpkin
spice is
part
of
what
makes
fall
special
for
them,
a
pumpkin
spice
latte
is
one
part
of
their
fall
tradition
“Pumpkin
Spice
Latte
at
Panera.
Oh
yeah,
I
need
one
of
those!
Bring
on
fall!
Looking
forward
to
bonfires
in
my
fire
pit
and
my
newly
refinished
fireplace.”
6%
PUMPKIN
TRADITIONALIST Loves
everything pumpkin
from
pumpkin
pie
to
lattes,
fall
is
just
an
excuse
to
get
their
fix
of
pumpkin
“Are
you
ready
for
a
Pumpkin
Spice
Latte!?!?!
Or
how
about
a
Pumpkin
Bar????
Well
tomorrow
they
both
will
be
available!!!!”

14. 14 |
Replicating
back
into
Brandwatch
PSL
PAMPERER
“morning
treat”
OR “Saved my
morning”
OR ((rough
OR bad
OR terrible*
OR awful
OR
stressful))
NEAR/4 (morning
OR
day
OR
week))
LATTE CHEMIST
((my OR I
OR Mine
OR
“made
a”)
NEAR/2f (organic
OR make
OR recipe
OR mixture))
OR homemade
OR “the
perfect”
OR
((coffee)
NEAR/3
(dessert
OR “sweet
tooth”)
FALL FANATIC
(I OR MY)
NEAR/3
(“love
fall”
OR “finally
here”
OR “the
season”
OR
autumn)
OR
((making
OR
made
OR
bake
OR
baked)
NEAR/4f (cake
OR pie
OR pastry))
PUMPKIN TRADITIONALIST
((pumpkin) AND (candle
OR products OR
cake
OR pie))
OR
“pumpkin
flavor”
OR ((“I
need
a”
OR “must
have”
OR “must
get”)
NEAR/3f
(latte))
We
conduct
a
careful
qualitative
analysis
of
persona
mentions
to
translate
the
topic
model
into
Brandwatch rules
• Allows
us
to
visualize
and
track
in
Brandwatch
• Create
each
persona
as
a
sub-­‐category
• Creating
the
persona
rules are
iteratively
written
Hypothetical
rules

15. 15 |
Custom
geo-­‐mapping
for
DMA’s
Persona
use
cases
Typing
tools
Scoring
conversation
relevance
IDENTIFYING
TARGET
SEGMENTS
Commentator
DIY PS
Flavor
Fall
(Season)
Treat
/
Reward
Focused
on
others
Traditional

16. 16
Next
level
– what
we’re
working
on
now
§ Ability
to
use
the
model
to
tag
incoming
mentions
in
Brandwatch
§ Determining
demographic
characteristics
from
language
§ Utilizing
topics
to
predict
outcomes

17. Thank
you
Bill
Harding
– Data
Scientist
billh@tahzoo.com
Colin
Rogers
– Direction
of
Content
Strategy
colinr@tahzoo.com