Beyond Buzz: On measuring a conversation Kate Niederhoffer, Ph.D Marc A. Smith, Ph.D Dachis Corporation Telligent Systems Web 2.0 4.1.09

Web 2.0 4.1.09

Why us? Kate Niederhoffer Ph.D UT Social Psychology BuzzMetrics/Nielsen Online, Measurement Science Dachis Corporation - Methodology, Social Business Design Marc Smith Ph.D UCLA Sociology Microsoft Research, Community Technologies Group Telligent Systems – “Harvest” reporting and analysis tools for social media platforms and systems Note: This is a conceptual address. We’re talking about ideas; each of our companies have distinct methodologies in place related to these concepts.

Kate Niederhoffer

Ph.D UT Social Psychology

BuzzMetrics/Nielsen Online, Measurement Science

Dachis Corporation - Methodology, Social Business Design

Marc Smith

Ph.D UCLA Sociology

Microsoft Research, Community Technologies Group

Telligent Systems – “Harvest” reporting and analysis tools for social media platforms and systems

Why are we here? Demonstrating the depth of buzz; ways to think about signal within vast universe. Going beyond buzz; learning more about individuals.

Demonstrating the depth of buzz; ways to think about signal within vast universe.

Going beyond buzz; learning more about individuals.

Why are we here? Highlighting the unique roles individuals play in communities that afford the conversation. Illustrating that aggregated relationships are network structures.

Highlighting the unique roles individuals play in communities that afford the conversation.

Illustrating that aggregated relationships are network structures.

Why now?

Blogs were all the rage In 2005, clients attracted by novelty: Simple question: What’s my buzz? - How much? - Good or bad? Incremental improvement: How “important” is it? - Are “Influencers” talking? - How many eyeballs exposed? - Engagement? However, all superficially measured; limited scope of what’s important: what kind of influence?

In 2005, clients attracted by novelty:

Simple question: What’s my buzz?

- How much?

- Good or bad?

Incremental improvement: How “important” is it?

- Are “Influencers” talking?

- How many eyeballs exposed?

- Engagement?

However, all superficially measured;

limited scope of what’s important: what kind of influence?

Blogs are now features Today’s “media” enable richer social interaction-- and, leave a path of data with more opportunities to capture depth Buzz levels, page views, followers, in isolation miss big picture Must take advantage context to tell whole story and capture value

Today’s “media” enable richer social interaction-- and, leave a path of data with more opportunities to capture depth

Buzz levels, page views, followers, in isolation miss big picture

Must take advantage context to tell whole story and capture value

Social networks are all the rage, but rarely do we think about social metrics We need to stop blackboxing : "When a machine runs efficiently, when a matter of fact is settled, one need focus only on its inputs and outputs and not on its internal complexity. Thus, paradoxically, the more science and technology succeed, the more opaque and obscure they become." - Bruno Latour Even if a conversation is running smoothly, we must figure out what makes it tick.

Central tenet: Social structure emerges from the aggregate of relationships (ties) among members of a population Phenomena of interest: Emergence of cliques and clusters from patterns of relationships Centrality (core), periphery (isolates), betweenness Methods: Surveys, interviews, observations, log file analysis, computational analysis of matrices Source: Richards, W. (1986). The NEGOPY network analysis program. Burnaby, BC: Department of Communication, Simon Fraser University. pp.7-16 Social Network Theory (Hampton &Wellman, 1999; Paolillo, 2001; Wellman, 2001)

Central tenet:

Social structure emerges from

the aggregate of relationships (ties)

among members of a population

Phenomena of interest:

Emergence of cliques and clusters

from patterns of relationships

Centrality (core), periphery (isolates),

betweenness

Methods:

Surveys, interviews, observations, log file analysis, computational analysis of matrices

Source: Richards, W. (1986). The NEGOPY network analysis program. Burnaby, BC: Department of Communication, Simon Fraser University. pp.7-16

Context of a conversation Relevance Role Mindset Ecosystem What is the pattern of connections? What is the dynamic, en masse? What else do we know about the individuals? Where’s the signal in the noise? Persona Person Environment Signal

What is the pattern of connections?

What is the dynamic, en masse?

What else do we know about the individuals?

Where’s the signal in the noise?

Context of a conversation Relevance Role Mindset Ecosystem Where’s the signal in the noise?

Where’s the signal in the noise?

Relevance today As a user, easy to relate to issues with pre-determined filters. As an enterprise, complexity increases. We don’t always know what we want to know!

As a user, easy to relate to issues with pre-determined filters.

As an enterprise, complexity increases.

Relevance: Which filters are in place to strengthen the signal? Identifying your filters can be inductive : What are people really saying? Which concepts differentiate the posts that mention you vs. posts that don't? All terms on your map have a correlation to the central concept; the closer a word appears to the center, the stronger the association.The groupings of terms indicate the dimensions of discussion: micro-conversations within a broader discussion. * Source: Nielsen Online, 2008

Identifying your filters can be inductive :

What are people really saying?

Which concepts differentiate the posts that mention you vs. posts that don't?

All terms on your map have a correlation to the central concept; the closer a word appears to the center, the stronger the association.The groupings of terms indicate the dimensions of discussion: micro-conversations within a broader discussion.

Relevance is multi-faceted Rather than looking at associations with , as compared to without , consider discussion this week as compared to discussion over the past year . Not what’s being said about her in a more recent timeframe, but instead when you control for what’s said about her in general, what pops? * Source: Nielsen Online, 2008

Rather than looking at associations with , as compared to without , consider discussion this week as compared to discussion over the past year .

Not what’s being said about her in a more recent timeframe, but instead when you control for what’s said about her in general, what pops?

Relevance - Summary Information can be visualized in so many different ways; don’t take it for granted. Listening can be limited if you’re exclusively looking for something in particular; broaden your net. Be inductive. Let the data speak for itself.

Information can be visualized in so many different ways; don’t take it for granted.

Listening can be limited if you’re exclusively looking for something in particular; broaden your net. Be inductive. Let the data speak for itself.

Context of a conversation Relevance Role Mindset Ecosystem What else can we know about the individuals?

What else can we know about the individuals?

Says Who?

Mindset By measuring the types of words used, we can tap into how people ‘slice’ their worlds. Linguistic style is closely tied to: Demographics (e.g. age, sex, class) Emotion (e.g. depression, deception) Cognitive style (e.g. complex thinking) Personality (e.g. Neuroticism) What else can we know about the person in conversation? e.g. Pennebaker, Mehl, Niederhoffer, 2003 Findings Linguistic Cues Are you self-oriented? Pronoun use: I and We Are you living in ‘the now’? Past, Present, Future tense What is your emotional tone? Positive vs. Negative Are you abstract or concrete? Articles: “a” vs. “the” Nouns vs. verbs

By measuring the types of words used, we can tap into how people ‘slice’ their worlds.

Linguistic style is closely tied to:

Demographics (e.g. age, sex, class)

Emotion (e.g. depression, deception)

Cognitive style (e.g. complex thinking)

Personality (e.g. Neuroticism)

When people make recommendations on blogs, is there something deeper going on? “ Got the next three PW/GS games for my birthday. And I am one happy gal, there was some stuff that I absolutely LOVED and I would definitely recommend the game to anyone who owns a PS3 regardless of its flaws -- which really were at their heart personal quibbles of mine so your mileage may vary. Plus, I cried like a b*$$ at the end. That's got to be saying something.”

Getting into the Engaged Mind Recommendations have: More pronouns: intimacy with both the brand/product/ service being recommended, and those to whom they’re recommending. More verbs : sharing experience more than discussion of concrete features. * all differences significant at p<.01 level

Recommendations have:

More pronouns: intimacy with both the brand/product/ service being recommended, and those to whom they’re recommending.

More verbs : sharing experience more than discussion of concrete features.

“ Invisible” language gives us clues about individuals, and groups

Changes in work atmosphere, captured in words Tausczik, Scholand, and Pennebaker, 2009 Engineers, economists programmers collaborating on economic simulations of disasters Complexity of thought (-) Cohesion (-) Work information (-) Negative emotion (+) Funding lost

Engineers, economists programmers collaborating on economic simulations of disasters

Complexity of thought (-)

Cohesion (-)

Work information (-)

Negative emotion (+)

Funding lost

“ Connected Age”: relationships are groundwork of work Social : niceties (lol), affirmations (cool), coordination (call), broad communication (http, thinking) Work : economic (production, supply), analytic (results, problem)

Mindset- Summary Language is a good way to go beyond the surface and better understand constituents without self- report biases (or effort). Metrics in the hands of users (yourselves) are helpful: know thyself, know how you’re perceived.

Language is a good way to go beyond the surface and better understand constituents without self- report biases (or effort).

Metrics in the hands of users (yourselves) are helpful: know thyself, know how you’re perceived.

Beyond thoughts and feelings, who comes to roost?

Context of a conversation Relevance Role Mindset Ecosystem What is the pattern of connections?

What is the pattern of connections?

Social Network Analysis with NodeXL : Identify different roles in social media spaces

Identify core groups in the network

Answer person Outward ties to local isolates Relative absence of triangles Few intense ties Reply Magnet Ties from local isolates often inward only Sparse, few triangles Few intense ties Distinguishing attributes:

Answer person

Outward ties to local isolates

Relative absence of triangles

Few intense ties

Reply Magnet

Ties from local isolates often inward only

Sparse, few triangles

Few intense ties

Distinguishing attributes: Answer person Outward ties to local isolates Relative absence of triangles Few intense ties Discussion person Ties from local isolates often inward only Dense, many triangles Numerous intense ties

Answer person

Outward ties to local isolates

Relative absence of triangles

Few intense ties

Discussion person

Ties from local isolates often inward only

Dense, many triangles

Numerous intense ties

Answer Person Signatures Discussion People

Spammer Discussion Starter Reply oriented Discussion Flame Warrior

Role – Summary Network awareness, like court vision enables strategic play. Know which positions/players are on your team. Social media behavior is differentiated. Rare (~.5-2%) roles are critical and must be cultivated. E.g. Clear and consistent signatures of an “Answer Person Light touch to numerous threads initiated by someone else Most ties are outward to local isolates Many more ties to small fish than big fish

Network awareness, like court vision enables strategic play. Know which positions/players are on your team.

Social media behavior is differentiated. Rare (~.5-2%) roles are critical and must be cultivated.

E.g. Clear and consistent signatures of an “Answer Person

Light touch to numerous threads initiated by someone else

Most ties are outward to local isolates

Many more ties to small fish than big fish

What is the mix in the neighborhood?

Context of a conversation Relevance Role Mindset Ecosystem What is the dynamic, en masse?

What is the dynamic, en masse?

Pajek without modification can sometimes reveal structures of great interest. The Ties that Blind?

Darwin Bell

Two “answer people” with an emerging 3 rd . Mapping Newsgroup Social Ties Microsoft.public.windowsxp.server.general

Research shows social media spaces vary and roles are present Adamic et al. WWW 2008

Ecosystem- Summary Social media is about collective action. A balance of roles and strategies is critical for a healthy/ successful collective good. Harvesting the common good takes many forms, and is the ultimate goal of social media.

Social media is about collective action.

A balance of roles and strategies is critical for a healthy/ successful collective good.

Harvesting the common good takes many forms, and is the ultimate goal of social media.

Why does this matter? This is not measurement for the sake of measurement; we need to measure conversations in order to manage social business. Measuring conversations is about measuring the context in which those conversations arise. Value is an intermediate step in calculating ROI. Moot to bypass it. Techniques from social science help capture “the immeasurable” in social media and the enterprise. The future of conversations- the enterprise being one-- is about cultivating ecologies of the right balance of relationships.

This is not measurement for the sake of measurement; we need to measure conversations in order to manage social business.

Measuring conversations is about measuring the context in which those conversations arise.

Value is an intermediate step in calculating ROI. Moot to bypass it.

Techniques from social science help capture “the immeasurable” in social media and the enterprise.

The future of conversations- the enterprise being one-- is about cultivating ecologies of the right balance of relationships.

Thank You [email_address] [email_address] Questions?

Additional Resources

How uniform are social media producing groups? Individuals Small Groups Variable Contribution Large Groups Uniform Large Groups Heterogeneous Variable Contribution Large Groups

Social Science Theory and Method Interactionist Sociology Central tenet Focus on the active effort of accomplishing interaction Phenomena of interest Presentation of self Claims to membership Juggling multiple (conflicting) roles Frontstage/Backstage Strategic interaction Managing one’s own and others’ “face” Methods Ethnography and participant observation (Goffman, 1959; Hall, 1990) Collective Action Dilemmas Central tenet Individual rationality leads to collective disaster Phenomena of interest Provision and/or sustainable consumption of collective resources Public Goods, Common Property, &quot;Free Rider” Problems, Tragedies Methods Surveys, interviews, participant observation, log file analysis, computer modeling (Axelrod, 1984; Hess, 1995; Kollock & Smith, 1996)

Interactionist Sociology

Central tenet

Focus on the active effort of accomplishing interaction

Phenomena of interest

Presentation of self

Claims to membership

Juggling multiple (conflicting) roles

Frontstage/Backstage

Strategic interaction

Managing one’s own and others’ “face”

Methods

Ethnography and participant observation

(Goffman, 1959; Hall, 1990)

Collective Action Dilemmas

Central tenet

Individual rationality leads to collective disaster

Phenomena of interest

Provision and/or sustainable consumption of collective resources

Public Goods, Common Property, &quot;Free Rider” Problems, Tragedies

Methods

Surveys, interviews, participant observation, log file analysis, computer modeling

(Axelrod, 1984; Hess, 1995; Kollock & Smith, 1996)