SOCIAL MEDIAMEASUREMENT 101A Day of aTONEmentMRA ANNUAL CONFERENCEJUNE 6, 2011Alan ChumleySenior Vice President@alanchumle...
ALAN CHUMLEY, SVP •   14 years in the industry •   Recovering PR practitioner:  ING, Bell Canada      R       i PR        ...
ABOUT CARMA •   Traditional & social  •   Media monitoring & analysis     M di        it i &    l i •   Largely human‐base...
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENTIN-ROOM STRETCH (5 mins)PART 2 (30 mins): TONE ALONE            i )PART 3: (35 m...
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma....
ISN’T• Not a Case Study• Not a mention of Old Spice in sight• Not a specific ‘how to’ / no one-size-fits all• Doesn’t focu...
ISN’T                                                           IS• Not a Case Study                                      ...
Every hour more than 500,000 newblog + micro-blog posts, status updates       micro blog postsand comments are posted 1615...
MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES                                                                ...
SOCIAL MEDIA IS A GAME CHANGER…                            …& A SOURCE OF ANXIETY           Top Down     Command & Control...
SOCIAL MEDIA IS A GAME CHANGER…                                    …& A SOURCE OF ANXIETY           Top Down              ...
SOCIAL MEDIA IS A GAME CHANGER…                                    …& A SOURCE OF ANXIETY           Top Down              ...
SOCIAL MEDIA IS A GAME CHANGER…                                       …& A SOURCE OF ANXIETY           Top Down           ...
SIMPLE STATUS QUOCounts:Quick Quantity counts:•Fans, Likes Follows, Etc.•Fans Likes, Follows Etc•Tantamount to reach in th...
LOT’S OF TOOLS OUT THEREBEWARE OF THE SILOs                                      AVOID USING (ONLY) THE NEW SHINEY FREEBIE...
REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE& NOT ALL CONVERSATIONS ARE RELEVANT                                            ...
REMEMBER: IT’S SOCIAL MEDIA•It’s not coverage•It’s not static It s•There isn’t an ‘audience’ of consumers•Prosumers not ju...
THINK BIGGER / INTEGRATE / CORRELATE  •Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all t...
THE MEASUREMENT MASHUP       1615 M Street, NW; Suite 750       Washington, DC 20036 USA Tel 1.202.842.1818       carma.co...
THINK BIGGER & BROADERMOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS                                                    ...
FOCUSING ON CONTENT ANALYSIS AND NETWORK ANALYSISAS A WAY OF MEASURING INFLUENCE POTENTIAL                                ...
The 7Cs                  THINKING MORE LIKE A SOCIOLOGIST THAN PUBLICISTCounting              What some call search, syndi...
THE 5Ps OF INFLUENCEPopular                          Visible, vocal, has a substantial following, reach. In‐bound links, t...
VARIETY OF OBJECTIVES & CAMPAIGN TYPES                              (NON-EXHAUSTIVE)Objectives                            ...
SOME THINKING ON AN APPROACHAN INDEX IS ONLY PART OF THE ANSWER             1615 M Street, NW; Suite 750             Washi...
COMPONENTS OF AN INDEX ?        Popularity  +  Relevance  +  Authority  +  Presence  +  Engagement  + Advocacy/Loyalty  + ...
INDICATORS DEFINED•   Cascade                                                             High          High      •Authori...
NETWORK ANALYSIS                                                Source: K. Ognyanova, UCLA, 2010                How are th...
(A QUICK ASIDE ON)   SENTIMENT AS MARKET RESEARCH PROXY / PREDICTORThink here about social media monitoring andanalysis as...
BUT REMEMBER THAT OBJECTIVES DRIVE METRICSObjectives                                                           Campaign Ty...
IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES         1615 M Street, NW; Suite 750         Washington,...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
AN INDEX WITH MOVABLE INDICATORS  FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES   Indicator     Popularity          ...
WHAT DOES THE INDEX INDICATE?•   It’s a relative measure of the quality of social media coverage:      – One post relative...
SO WHAT?WHAT ABOUTCONVERSION & ROI?1615 M Street, NW; Suite 750Washington, DC 20036 USA Tel 1.202.842.1818carma.com
WE’RE BACK TO THE PYRAMID                              THE TOP THIS TIME         1615 M Street, NW; Suite 750         Wash...
OUTCOMES: KINDS OF CONVERSION   SETTING THE STAGE FOR CORRELATION                                                         ...
2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITEWhere were they? What drove them to your site?What did they do once...
SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICSExport Data to Your Web Analytics•#of posts by date•# of posts with key messa...
LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage                                NotesJust graph ‘er up   Plot # of art...
LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage                                NotesJust graph ‘er up   Plot # of art...
LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage                                NotesJust graph ‘er up   Plot # of art...
TECH INDUSTRY EXAMPLE  Correlating opinion with  CARMA IQ (favorability +  reach)         1615 M Street, NW; Suite 750    ...
AUTO INDUSTRY EXAMPLE    Monthly Sales Results vs. Four Month CARMA Favorability:       Monthly Sales to Four Previous Mon...
Toyota’s Heat Map:CARMA Media IQ Metrics by Various Time Frames Correlated toMonthly Sales Results      y          1615 M ...
UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs                   ROI                                             ...
THOSE USING TXT ANALYTICS ARE MEASURING ‘ROI’ NOT ‘roi’   Source: Text Analytics 2009: U   S       T t A l ti 2009 User P ...
SOCIAL MEDIA ROI CALCULATOR(S)                          THE GOOD & THE BAD          1615 M Street, NW; Suite 750          ...
SOCIAL MEDIA BOOK BAG•   Social Media Metrics:            • Jim Sterne, (@jimsterne)•   Social Media ROI:            • Oli...
PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENTIN-ROOM STRETCH (5 mins)PART 2 (30 mins): TONE ALONEPART 3: (35 mins): REAL DA...
AGENDA FOR PART 2                                              DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is it importa...
AGENDA FOR PART 2                                                         DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is...
AGENDA FOR PART 2                                                         DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is...
I AM NOT                                                                       IAMA COMPUTATIONAL LINGUISTA SEMANTICIANCOM...
I AM NOT                                                                                      IAMA COMPUTATIONAL LINGUIST ...
IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES         1615 M Street, NW; Suite 750         Washington,...
WHY IS IT IMPORTANT?                                    WHY SHOULD WE CARE?Questions for biz & gov’t.:•What are people say...
SENTIMENT ANALYSIS AS MARKET RESEARCH PROXY / PREDICTORThink here about social media monitoring andanalysis as:•Real time ...
APPLICATIONS                                                     Call Centre Notes?       1615 M Street, NW; Suite 750    ...
SITUATING SENTIMENT ANALYSIS                                                    AUTOMATED                                 ...
AUTOMATED SENTIMENT ANALYSIS                                                   DEFINEDSentiment analysis or opinion mining...
EXTRACTING ATTITUDES                                     TURNING ATTITUDES INTO DATAGenerally speaking, sentiment analysis...
HUMANS OR MACHINES OR BOTH?ALGORITHM                                                                     AL GORITHM Can it...
WHY AUTOMATION?                     Time Spent                  Analyzing Content                      y g                ...
WHY AUTOMATION?                     Time Spent                  Analyzing Content                      y g                ...
THE OUTPUT                                                     Entity E t ti                                              ...
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES                                                       • Words/phr...
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES                                                       • Words/phr...
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES                                                       • Words/phr...
TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES                                                       • Words/phr...
AUTOMATED TEXT ANALYTICS VENDORS        1615 M Street, NW; Suite 750        Washington, DC 20036 USA Tel 1.202.842.1818   ...
1st GENERATION vs. 2nd GENERATION AUTOMATED SENTIMENT                                                     Conceptual Searc...
COMPLICATIONS & PROBLEMS w/ 1ST GEN AUTOMATED APPROACHES •Sentiment may be of interest at multiple levels:      •Document ...
CONTEXT IS KEY       1615 M Street, NW; Suite 750       Washington, DC 20036 USA Tel 1.202.842.1818       carma.com       ...
ANALYSING THE “CONNECTIVE TISSUE” OF A SENTENCE       1615 M Street, NW; Suite 750       Washington, DC 20036 USA Tel 1.20...
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Social Media Measurement 101 + Day of aTONEment
Upcoming SlideShare
Loading in...5
×

Social Media Measurement 101 + Day of aTONEment

1,459

Published on

3-hour presentation at MRA\'s Annual Conference, June 6, 2011. Part 3 by Malcolm Deleo @ Netbase

0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,459
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
18
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Social Media Measurement 101 + Day of aTONEment

  1. 1. SOCIAL MEDIAMEASUREMENT 101A Day of aTONEmentMRA ANNUAL CONFERENCEJUNE 6, 2011Alan ChumleySenior Vice President@alanchumley / @CARMA_Tweets@ l h l @CARMA T t1615 M Street, NW; Suite 750Washington, DC 20036 USA Tel 1.202.842.1818carma.com
  2. 2. ALAN CHUMLEY, SVP • 14 years in the industry • Recovering PR practitioner:  ING, Bell Canada  R i PR titi ING B ll C d • Former Director of Measurement @ Hill & Knowlton • Masters in Communications & Culture: • Research foci:   • media effects & uses  • reception analysis reception analysis  • audience studies • University‐level adjunct instructor of PR measurement • LinkedIn early 2006 Li k dI l 2006 • Blogging / Tweeting since early 2007 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 2 @alanchumley / #mra_ac
  3. 3. ABOUT CARMA • Traditional & social  • Media monitoring & analysis M di it i & l i • Largely human‐based….on an automation journey • In biz since 1984 • Head office here in DC • 14 offices in 11 countries 14 offices in 11 countries • Clients: • PR f lk i bi PR folk in big organizations i ti • Market research folks in big organizations are starting to emerge 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 3 @alanchumley / #mra_ac
  4. 4. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENTIN-ROOM STRETCH (5 mins)PART 2 (30 mins): TONE ALONE i )PART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com
  5. 5. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENT 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com
  6. 6. ISN’T• Not a Case Study• Not a mention of Old Spice in sight• Not a specific ‘how to’ / no one-size-fits all• Doesn’t focus on a singular or few socialmedia platforms—looks at all holistically 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  7. 7. ISN’T IS• Not a Case Study • An emerging / experimental philosophy / approach / filter / lens through which to look at measuring social media• Not a mention of Old Spice in sight • A high level connected / systems way of• Not a specific ‘how to’ / no one-size-fits all approaching the problem• Doesn’t focus on a singular or few social • About communities not an audience b dmedia platforms—looks at all holistically • Beyond simple counts and sentiment • More sociologist than publicist • Influence potential 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  8. 8. Every hour more than 500,000 newblog + micro-blog posts, status updates micro blog postsand comments are posted 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com
  9. 9. MEASUREMENT ACCOUNTS FOR 5 / 7 TOP SOCIAL MEDIA CHALLENGES Why is this the case? Gap between leap of faith true believers and C-suite’s need for metrics to validate the time/$? $ Source: Harvard Business Review: the New Conversation: Taking Social Media from Talk to Action Surveyed 2100 Harvard Biz Review subscribers 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 9 @alanchumley / #mra_ac
  10. 10. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY Top Down Command & Control of the Message is Gone• Transparency demanded• Dialogue vs. Monologue• Many-to-many not one-to- many• Conversation & community NOT (strictly) coverage• About relationships• Relationship trust is co- created by the organization and its communities of interest via conversation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  11. 11. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY Top Down There is no Passive Command & Control Ready-to-Consume Audience’ of the Message is Gone• Transparency demanded • Prosumers not just consumers• Dialogue vs. Monologue • Actively engaged in creating and re-creating conversations and g• Many-to-many not one-to- communities many • Challenge in tracking &• Conversation & community measuring something as NOT (strictly) coverage nebulous as a community• About relationships• Relationship trust is co- created by the organization and its communities of interest via conversation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  12. 12. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY Top Down There is no Passive Disciplinary Lines Command & Control Ready-to-Consumer ‘Audience’ are Eroding of the Message is Gone • PA, PR, Marketing, CRM,• Transparency demanded • Prosumers not just consumers Customer Service etc.• Dialogue vs. Monologue • Actively engaged in creating and • Customer touch points re-creating conversations and g• Many-to-many not one-to- communities • Integrated cross-discipline many re: strategy • Challenge in tracking &• Conversation & community measuring something as • Integrate cross-discipline NOT (strictly) coverage nebulous as a community re: measurement• About relationships• Relationship trust is co- created by the organization and its communities of interest via conversation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  13. 13. SOCIAL MEDIA IS A GAME CHANGER… …& A SOURCE OF ANXIETY Top Down There is no Passive Disciplinary Lines Command & Control Ready-to-Consumer ‘Audience’ are Eroding of the Message is Gone • PA, PR, Marketing, CRM,• Transparency demanded • Prosumers not just consumers Customer Service etc.• Dialogue vs. Monologue • Actively engaged in creating and • Customer touch points re-creating conversations and g• Many-to-many not one-to- communities • Integrated cross-discipline many re: strategy • Challenge in tracking &• Conversation & community measuring something as • Integrate cross-discipline NOT (strictly) coverage nebulous as a community re: measurement• About relationships• Relationship trust is co- C-Suite C Suite is Intrigued but Anxious / Want Data to: created by the • Show why and how they should… organization and its • Demonstrate the value of having done so… communities of interest via conversation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  14. 14. SIMPLE STATUS QUOCounts:Quick Quantity counts:•Fans, Likes Follows, Etc.•Fans Likes, Follows Etc•Tantamount to reach in the offline or 1.0 world•Reach does not equal InfluenceQuality:•Quick automated read on sentiment: but concerns re: accuracyRankings:•Visibility, popularity, traffic, link love Visibility,•Technorati for blogs•NOT RECOMMENDED: Twitrratr, Klout,•BETTER: Twendz, Traackr, Twittalyzer, Twinfluence•They rarely qualify people or content (the message) The rarel q alif•The message AND the medium are the measure 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 14 @alanchumley / #mra_ac
  15. 15. LOT’S OF TOOLS OUT THEREBEWARE OF THE SILOs AVOID USING (ONLY) THE NEW SHINEY FREEBIES •Isolated / silo’d approach •No one tool on it’s own is sufficient—not even one paid tool it s sufficient not •Have to connect the dots with a more relational, systems approach Would you like your job performance (read: raise/bonus) to be based only on a 1615 M Street, NW; Suite 750 Klout Score, for example? Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  16. 16. REACH SHOULD BE A RELATIVE NOT AN ABSOLUTE& NOT ALL CONVERSATIONS ARE RELEVANT Off topic / Irrelevant Relevant R l t Relevant “ ” R l t “-” Relevant “+” Neutral 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 16 @alanchumley / #mra_ac
  17. 17. REMEMBER: IT’S SOCIAL MEDIA•It’s not coverage•It’s not static It s•There isn’t an ‘audience’ of consumers•Prosumers not just consumers: •simultaneously receiving, interpreting, co-opting, re-appropriating and creating content•It’s more many-to-many di l It’ t dialogue th one-to-many monologue than t l•So, traditional off line ways of measuring ‘coverage’ don’t cut it•They don’t adequately account for the multi-dimensionality and dynamicnature of a conversation and a community of interest in a network•We need to think bigger gg 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 17 @alanchumley / #mra_ac
  18. 18. THINK BIGGER / INTEGRATE / CORRELATE •Think Bigger / Broader Beyond PR’s Historical Silo –social Media’s Pushing us all that Way  •Think about all customer‐facing touch points •Understand PR / Marketing / Business Metrics   g p •Integrate Cross‐Discipline •Report on the Critical Few •Focus on /correlate with Outcomes •Close the loop / connect the dotsSource: W b Sh d i k Measurement & Strategy PracticeS Weber-Shandwick M t St t P ti Source: Oliver Blanchard, S i l M di ROI S Oli Bl h d Social Media 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 18 @alanchumley / #mra_ac
  19. 19. THE MEASUREMENT MASHUP 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 19 @alanchumley / #mra_ac
  20. 20. THINK BIGGER & BROADERMOVE BEYOND ‘MOMENT-IN-TIME’ or ‘VANITY’ METRICS Systems or N t S t Networked k d Thinking: Interdependent relationships between p those in a network Social Capital: Connections within and between those in a social network as a source of civic engagement among a community leading to 1615 M Street, NW; Suite 750 cohesion; that has ‘value’ Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 20 @alanchumley / #mra_ac
  21. 21. FOCUSING ON CONTENT ANALYSIS AND NETWORK ANALYSISAS A WAY OF MEASURING INFLUENCE POTENTIAL Systems or N t S t Networked k d Thinking: Interdependent relationships between p those in a network Social Capital: Connections within and between those in a social network as a source of civic engagement among a community leading to 1615 M Street, NW; Suite 750 cohesion; that has ‘value’ Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 21 @alanchumley / #mra_ac
  22. 22. The 7Cs THINKING MORE LIKE A SOCIOLOGIST THAN PUBLICISTCounting What some call search, syndication and site; stuff we can count like traffic, popularity, fans, followers,  retweets, etc. Content Analysis that is.  Using content analysis, ideally a hybrid of some automation for volume and speed and  y g y , y y p human analysts for context, nuance and depth to look for how your brand, messaging, topics, issues,  stakeholder voices are portrayed; the inclusiveness, accuracy and tone of the conversations. Conversations And conversationships recognizing that social media is not strictly another communications channel  through which to get word out or reach and influence.  It is more about genuinely engaged dialogue  than a simple corporate monologue; not only a vertical, one to many conversation between an  than a simple corporate monologue; not only a vertical, one‐to‐many conversation between an organization and stakeholders but a lateral, many‐to‐many conversation among stakeholders.     Cohesion The extent to which folks are agreeing with you, your position, your brand essences.  The extent to  which they are agreeing with each other and coalescing around and advocating a core theme, idea, or  some call to action either in support or opposition to your organization s position.  some call to action either in support or opposition to your organization’s positionCommunity The extent to which a core group of people with common interests are gathering and growing.   Measuring the change in community size, and volume and nature of chatter over timeConnectedness A look at how inter‐connected the highly engaged/key influencers/advocates are and how centrally  located they are in both the network and dialogue.    (See Appendix for the 5Ps of influence)Conversion Stay tuned.  More to follow.   1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 22 @alanchumley / #mra_ac
  23. 23. THE 5Ps OF INFLUENCEPopular  Visible, vocal, has a substantial following, reach. In‐bound links, trackbacks, subscribers,  bookmarks, followers, friends, views, listens, saves, downloads, etc.Polarized in tone Neutrality does little to drive influence way or the other. A clearly positive or  negative view will polarize readers/followers and is more likely to drive cohesion and  mobilize advocates and have those advocates coalesce around a core theme, idea, or call  to action.Prolific / Relevant /  Raw author contribution and # of on‐topic, related postsFrequentProminent /  Are they an idea starter or spreader; source or spider? They may be prolific but are they  prominent? Are they highly inter related inter connected and centrally located in the Are they highly inter‐related, inter‐connected, and centrally located in the Authoritative network? How engaged is this person’s following in a dialogue? How much dialogue is  there and what is its nature? Here we need to recognize, though, that authority is  contextual and topical. One might be an authority on PR measurement but not on 18th  century Russian literature.Promoter / Loyal  / l How many of th f ll H f the followers/commentators active contributors advocating, endorsing,  / t t ti t ib t d ti d i advancing (or the opposite) your position? Are they adding links, tags. Is the nature of the Advocate language they are using inter‐connective, expanded, clarifying, reinterpreting? RTs, digs,  fans, votes, buzz‐ups, up/downloads, shares, likes, invites, favorites, embeds. (More active  than the metrics in popularity) 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 23 @alanchumley / #mra_ac
  24. 24. VARIETY OF OBJECTIVES & CAMPAIGN TYPES (NON-EXHAUSTIVE)Objectives Campaign Types• Visibility • Push (get bloggers to review / write)• Visits • Pull  (drive visitors on‐line)• Volume • CSR• Velocity • Product Launch Product Launch• Awareness • Product Engagement• Message delivery • Marketing• Link love • Blogger Outreach• Dialogue vs. Monologue Dialogue vs Monologue • Advocacy• Generate dialogue / comments • Issues Management• Feedback • Community building• Using, sharing, pass‐a‐long • Customer relations Customer relations• Embedding of interactive content • Thought leadership • Diggs, Delicious tags, stumbles, etc. • Executive profiling• Votes • Etc., etc., etc.  • Etc., etc., etc.   Etc., etc., etc. 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 24 @alanchumley / #mra_ac
  25. 25. SOME THINKING ON AN APPROACHAN INDEX IS ONLY PART OF THE ANSWER 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 25 @alanchumley / #mra_ac
  26. 26. COMPONENTS OF AN INDEX ? Popularity  +  Relevance  +  Authority  +  Presence  +  Engagement  + Advocacy/Loyalty  + Cascade x  (+/‐) Sentiment = Influence Potential• Based on an assumption that the components above are drivers of on-line influence, potential action• They can be measured individually, but it’s important to measure them collectively as they are inter-related• The index is intended as a measure of potential to influence not actual real influence that is another influence—thatresearch project all together.• All components are important (but not equal) drivers of potential influence on-line p p ( q ) p 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 26 @alanchumley / #mra_ac
  27. 27. INDICATORS DEFINED• Cascade High  High  •Authority • Citation analysis / ripple index authority   authority  •Of both poster and commentators • Network analysis low  AND  •Idea starters or spreaders? relevance relevance •Degree of inter connectedness / inter relatedness •Degree of inter‐connectedness / inter‐relatedness • Volume & velocity of message spread across the  V l & l i f d h network •Connected but respected? • Inter‐connectedness / inter‐relationships among: •How central is the influencer to the network? Low  High  •Following: follower • all authority  relevance  • Only the influentials AND  low  •RTs/1000 followers relevance authority• Advocacy (active) y( ) • Are social media users advancing our cause? • Relevance Rele ance • % containing links • On topic? • % active contributors • Author contribution:  how often / prolific? • Tags added • Topical / textual / temporary• Engagement (more coming) Engagement (more coming) • Sentiment (more coming) S ti t( i ) • Dialogue?  2‐way?  How much? How good?   • How many commentators?  Repeats?  Nature of  • Multi‐level, multi‐author tone the language • Post‐level, theme level, sentence level • Network analysis:  inter‐connected/related  • Gaps  in tone in originating content and resulting  discussion• Entity Presence Entity Presence • Present?  Prominent?  Frequent? • Popularity (passive)   • # of + mentions • Competitive share of original post • ‘Reach’  ? • Competitive share of resulting discussion • Voice of the consumer stuff we can count • Attribution recognition • Links, votes, shares, recommends, subscribers,  registrations, tweets, RTs, followers, fans, friends,  diggs, shares, likes, fav s, upvotes, buzzups,  diggs shares likes fav’s upvotes buzzups embeds, tags, wish list saves, forwards, invites,   etc. 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 27 @alanchumley / @CARMA_Tweets
  28. 28. NETWORK ANALYSIS Source: K. Ognyanova, UCLA, 2010 How are these changing over time? Might be part of many systems / networks? Linking networks to networks 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / @CARMA_Tweets
  29. 29. (A QUICK ASIDE ON) SENTIMENT AS MARKET RESEARCH PROXY / PREDICTORThink here about social media monitoring andanalysis as:•Real time situational awareness•Listening at the point of need•Listening close to the moment of influenceA real-time digital form of or proxy for marketresearch and CRM:Mining conversations to tease out how people‘feel’ about products/services/issues/causesand perhaps even:1) their intended (reported) behavior and whatmotivates them to buy, (as cited in theconversations), and2) customer satisfaction3)their actual behavior tracked via CRM 2.0 andWeb 2.0 analytics y 1615 M Street, NW; Suite 750 Source: Crimson Hexagon Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 29 @alanchumley / #mra_ac
  30. 30. BUT REMEMBER THAT OBJECTIVES DRIVE METRICSObjectives Campaign Types• Visibility • Push (get bloggers to review / write)• Visits • Pull  (drive visitors on‐line)• Volume • CSR• Velocity • Product Launch Product Launch• Awareness • Product Engagement• Message delivery • Marketing• Link love • Blogger Outreach• Dialogue vs. Monologue Dialogue vs Monologue • Advocacy• Generate dialogue / comments • Issues Management• Feedback • Community building• Using, sharing, pass‐a‐long • Customer relations Customer relations• Embedding of interactive content • Thought leadership • Diggs, Delicious tags, stumbles, etc. • Executive profiling• Votes • Etc., etc., etc.  • Etc., etc., etc.   Etc., etc., etc. 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 30 @alanchumley / #mra_ac
  31. 31. IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 31 @alanchumley / #mra_ac
  32. 32. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 32 @alanchumley / #mra_ac
  33. 33. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 33 @alanchumley / #mra_ac
  34. 34. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 34 @alanchumley / #mra_ac
  35. 35. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 35 @alanchumley / #mra_ac
  36. 36. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 36 @alanchumley / #mra_ac
  37. 37. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 37 @alanchumley / #mra_ac
  38. 38. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 38 @alanchumley / #mra_ac
  39. 39. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 39 @alanchumley / #mra_ac
  40. 40. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 40 @alanchumley / #mra_ac
  41. 41. AN INDEX WITH MOVABLE INDICATORS FLEXIBLE ENOUGH TO ACCOUNT FOR DIFFERENT OBJECTIVES Indicator Popularity Relevance Authority Brand Engagement Advocacy Cascade Sentiment Presence Score 10 10 10 10 10 10 10 10 Weight 1 2 3 4 5 6 7 (@ d f lt) default) Popularity + Relevance + Authority + Presence + Engagement + Advocacy + Cascade X Sentiment = Influence Potential• Each of the above indicators can be adjusted to account for different types of campaigns & contexts • The weights would change.  The scores don’t.      • For an organization, new to social media, perhaps they might only want to establish an on‐line presence• So we’d dial or slide the weights of all but the popularity indicator down to zero Has a significantly scaled back / rudimentary version of this. This is inspired by but significantly expands on R6 thinking. p y g y p g 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 41 @alanchumley / #mra_ac
  42. 42. WHAT DOES THE INDEX INDICATE?• It’s a relative measure of the quality of social media coverage: – One post relative to another and over time  – Many posts relative to a number of other similar clusters of postings• It is a measure of both output and outtake, but not outcome – Output:  how much and how good? – Outtake:  how engaged, inter‐connected, inter‐related, volume & velocity of network /  community spread – Outtake:  advocacy – it does not claim to be a measure of outcome or impact d l b f• It’s a proxy for on‐line influence; it’s about potential to influence – to measure actual influence, we’ve have to survey• In a diagnostic, benchmarking sense the index can help inform an organization s social media  In a diagnostic benchmarking sense the index can help inform an organization’s social media engagement strategy if tracked over time• It’s tactical on the fly for mid‐campaign course correction and looking back over the long term• It’s strategic looking forward: helps shape communications recommendations• This data can be correlated with tangible outcomes—CARMA Connect.  • This data can be plugged into a market mix model to show causality…isolate for PR’s unique  contribution to the marketing mix.      1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 42 @alanchumley / #mra_ac
  43. 43. SO WHAT?WHAT ABOUTCONVERSION & ROI?1615 M Street, NW; Suite 750Washington, DC 20036 USA Tel 1.202.842.1818carma.com
  44. 44. WE’RE BACK TO THE PYRAMID THE TOP THIS TIME 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 44 @alanchumley / #mra_ac
  45. 45. OUTCOMES: KINDS OF CONVERSION SETTING THE STAGE FOR CORRELATION • Conversion to a tangible like web traffic, sales, donations, awareness, opinion,Micro  (steps toward destination) Macro (Destination) usage, brand preference, likelihood toUsually an intermediate means to an end The end try/buy/recommend.Non‐transactional (Engagement) Transactional • Could also mean conversion toward anyNon‐financial Financial measurable marketing communication or public affairs, issues, advocacy-basedGet to x clicks in a site, spend x time on  , p Sale, donation,  , , objective. objectivesite, download, lead generated, watch a  (paid) subscription video, created account, contacted us etc.  etc.   • Here we could / should also be looking at the extent to which the quantity and q q y quality y of social media activity correlates with these outputs. 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 45 @alanchumley / #mra_ac
  46. 46. 2 WEEKS IN THE SOCIAL MEDIA ECOSYSTEM…THEN…YOUR WEBSITEWhere were they? What drove them to your site?What did they do once on your site? What are they ‘worth’ to you? 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 46 @alanchumley / #mra_ac
  47. 47. SOCIAL MEDIA PERFORMANCE DATA + WEB ANALYTICSExport Data to Your Web Analytics•#of posts by date•# of posts with key message included # of posts with key message included•# of post with brand mentioned X or more times•# posts X average favorability by date•# posts X average favorability X reach by date p g y y OrIntegrate web analytics intoyour social mediamonitoring platform 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 47 @alanchumley / #mra_ac
  48. 48. LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage NotesJust graph ‘er up Plot # of articles vs.  •No indication of  web traffic on a graph relationship other  than lines that  might trend  together •Could be a fluke Source: Olivier Blanchard, Social Media ROI 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 48 @alanchumley / #mra_ac
  49. 49. LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage NotesJust graph ‘er up Plot # of articles vs.  •No indication of  web traffic on a graph relationship other  than lines that  might trend  together •Could be a flukeCorrelation •“R score” •Showing there is  S t t •Some stats  some relationship  l ti hi •You can do this in  beyond coincidence excel •Pearson’s Product  •Not ‘proving’  Moment media exclusively  responsible for sales Source: Olivier Blanchard, Social Media ROI 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 49 @alanchumley / #mra_ac
  50. 50. LEVELS OF LINKING OUTPUT TO OUTCOMESLevel of Linkage NotesJust graph ‘er up Plot # of articles vs.  •No indication of  web traffic on a graph relationship other  than lines that  might trend  together •Could be a flukeCorrelation •“R score” •Showing there is  S t t •Some stats  some relationship  l ti hi •You can do this in  beyond coincidence excel •Pearson’s Product  •Not ‘proving’  Moment media exclusively  responsible for sales Source: Olivier Blanchard, Social Media ROICausality •Heavy duty stats •Isolates  •You so can’t do this in  ( (statistically y excel ‘proves’) for PR’s  •Sophisticated  unique contribution modeling to the  •Market Mix Modeling communications  •Multivariate  mix Regression Analysis Regression Analysis •Ton ‘o data, time, $ 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 50 @alanchumley / #mra_ac
  51. 51. TECH INDUSTRY EXAMPLE Correlating opinion with CARMA IQ (favorability + reach) 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 51 @alanchumley / #mra_ac
  52. 52. AUTO INDUSTRY EXAMPLE Monthly Sales Results vs. Four Month CARMA Favorability:  Monthly Sales to Four Previous Months’ Favorability 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 52 @alanchumley / #mra_ac
  53. 53. Toyota’s Heat Map:CARMA Media IQ Metrics by Various Time Frames Correlated toMonthly Sales Results y 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 53 @alanchumley / #mra_ac
  54. 54. UPPERCASE ‘ROI’ vs. lowercase ‘roi’ & Other Return-Offs ROI roi MBA PR Financial Non‐financial Non financial Macro Micro (not all conversion events are ROI) A mathematical equation  Usually conceptual only; an equation in words only; can’t actually  that works that works be calculated; something the industry tosses around too liberally be calculated; something the industry tosses around too liberally A business metric All‐too‐often a media or audience metric Direct, clear A loose proxy or an intermediate metric (return on expectation,  return on objective, return on engagement) return on objective return on engagement) Source: Olivier Blanchard, Social Media ROI 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 54 @alanchumley / #mra_ac
  55. 55. THOSE USING TXT ANALYTICS ARE MEASURING ‘ROI’ NOT ‘roi’ Source: Text Analytics 2009: U S T t A l ti 2009 User P Perspectives on S l ti ti Solutions and P id d Providers 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  56. 56. SOCIAL MEDIA ROI CALCULATOR(S) THE GOOD & THE BAD 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  57. 57. SOCIAL MEDIA BOOK BAG• Social Media Metrics:   • Jim Sterne, (@jimsterne)• Social Media ROI:   • Olivier Blanchard (@thebrandbuilder)• Social Media Analytics:  Effective Tools for Building, Interpreting & Using Metrics:   • Marshal Sponder (@webmetricsguru)• Measure What Matters:   M Wh M • Katie Paine (@kdpaine)• Using Web Analytics to Measure Impact of Earned Online Media…: g y p • Seth Duncan (www.instituteforpr.org) 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 57 @alanchumley / #mra_ac
  58. 58. PART 1 (1.5 hrs): SOCIAL MEDIA MEASUREMENTIN-ROOM STRETCH (5 mins)PART 2 (30 mins): TONE ALONEPART 3: (35 mins): REAL DATA WITH MALCOM DE LEO NETBASE 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com
  59. 59. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is it important/ why should we care?•Sentiment as a Proxy / Predictor•Applications•Humans or Machines or Both? 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 59 @alanchumley / #mra_ac
  60. 60. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is it important/ why should we care?•Sentiment as a Proxy / Predictor•Applications •Why Automation?•Humans or Machines or Both? •Situating Automated S S Sentiment Analysis •Defining Automated Sentiment Analysis •Output •Taxonomies etc. •Automated Vendors •Complications p 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 60 @alanchumley / #mra_ac
  61. 61. AGENDA FOR PART 2 DEEP DIVE ON SENTIMENT•Sentiment Analysis•Why is it important/ why should we care?•Sentiment as a Proxy / Predictor•Applications •Why Automation?•Humans or Machines or Both? •Situating Automated S S Sentiment Analysis •Defining Automated Sentiment Analysis •Output •Human vs. Automated vs •Taxonomies etc. •Human-based vendors •Automated Vendors •Human-Based Sentiment Analysis •Complications p •Favorability not (just) Sentiment •Humans supplemented by automation Linking outputs to outcomes revisited 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 61 @alanchumley / #mra_ac
  62. 62. I AM NOT IAMA COMPUTATIONAL LINGUISTA SEMANTICIANCOMPUTER ENGINEERCOMPUTER SCIENTISTLOGICIANPSYCHOLINGUISTBUT…CHECKBUT CHECK OUT SETH GRIMES : GRIMES’:Text Analytics 2009: User Perspectives on Solutions and ProvidersA number of excellent presentations on LinkedIn / Slide ShareI’ve borrowed liberally from both for this Part 2 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 62 @alanchumley / #mra_ac
  63. 63. I AM NOT IAMA COMPUTATIONAL LINGUIST A RECOVERING PR FLACK CO G CA SEMANTICIAN PR MEASUREMENT MOUTHPIECECOMPUTER ENGINEER WITH AN INTEREST IN MONITORING +COMPUTER SCIENTIST MINING ONLINE CONVERSATIONSLOGICIANPSYCHOLINGUISTBUT…CHECKBUT CHECK OUT SETH GRIMES : GRIMES’: Sentiment through the eyesText Analytics 2009: User Perspectives on Solutions and ProvidersA number of excellent presentations on LinkedIn / Slide Share of a simpletonI’ve borrowed liberally from both for this Part 2 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 63 @alanchumley / #mra_ac
  64. 64. IMAGINE A DIFF’T. PYRAMID FOR DIFF’T. CAMPAIGN TYPES / OBJECTIVES 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 64 @alanchumley / #mra_ac
  65. 65. WHY IS IT IMPORTANT? WHY SHOULD WE CARE?Questions for biz & gov’t.:•What are people saying? Wh t l i ?•What’s trending?•How has opinion about x changed?•Does it d e c a e to c a e oes differ channel o channel•How has sentiment correlated withnews / marketing / events etc.?•Can we link opinions to transactions? Computers are trading stock based on automated sentiment analysis of online content 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 65 @alanchumley / #mra_ac
  66. 66. SENTIMENT ANALYSIS AS MARKET RESEARCH PROXY / PREDICTORThink here about social media monitoring andanalysis as:•Real time situational awareness•Listening at the point of need•Listening close to the moment of influenceA real-time digital form of or proxy for marketresearch and CRM:Mining conversations to tease out how people‘feel’ about products/services/issues/causes feeland perhaps even:1) their intended (reported) behavior and whatmotivates them to buy, (as cited in theconversations), and ti ) d2) their actual behavior tracked via CRM 2.0and Web 2.0 analytics 1615 M Street, NW; Suite 750 Source: Crimson Hexagon Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 66 @alanchumley / #mra_ac
  67. 67. APPLICATIONS Call Centre Notes? 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 67 @alanchumley / #mra_ac
  68. 68. SITUATING SENTIMENT ANALYSIS AUTOMATED Text analytics: •Applies linguistic +/or statistical Artificial Intelligence techniques to extract concepts and patterns Text A l ti T t Analytics •Transforms unstructured information into data for analysis •Unlocks meaning and relationships in large volumes of information Sentiment Analysis •Adds semantic understanding of: •Sentiment •Opinions Opinions •Attitudes •Entities •Concepts •Facts Facts 1615 M Street, NW; Suite 750 •Abstract attributes: “expensive,” Washington, DC 20036 USA Tel 1.202.842.1818 carma.com “comfortable” 68 @alanchumley / #mra_ac
  69. 69. AUTOMATED SENTIMENT ANALYSIS DEFINEDSentiment analysis or opinion mining refers to the application of: •Natural language processing Linguistics Comp Science •Computation linguistics p g •Text analytics……to identify and extract subjective information in content. StatisticsNatural Language Processing • Grounded in statistical machine learning • Much overlap with…Computational LinguisticsComputational Linguistics • Statistical or rule‐based modeling of languageText Analytics • Turning text into data • Lexical analysis:  word frequencies • Entity recognition • Pattern recognition • Tagging/ annotation Tagging/ annotation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com @alanchumley / #mra_ac
  70. 70. EXTRACTING ATTITUDES TURNING ATTITUDES INTO DATAGenerally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer with respect to some topic or the overall tonality of a document.The attitude may be his or her: • judgment or evaluation • affective state (emotional state of the author when writing), or • the intended emotional communication (emotional effect the author wishes to have on the reader). 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 70 @alanchumley / #mra_ac
  71. 71. HUMANS OR MACHINES OR BOTH?ALGORITHM AL GORITHM Can it be over ridden? Does it learn? THE ALGORITHMICS 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 71 @alanchumley / #mra_ac
  72. 72. WHY AUTOMATION? Time Spent Analyzing Content y g w/out automation Time Spent on Ti S Insights & Implications w/ automation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 72 @alanchumley / #mra_ac
  73. 73. WHY AUTOMATION? Time Spent Analyzing Content y g w/out automation Time Spent on Ti S Insights & Implications w/ automation 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 73 @alanchumley / #mra_ac
  74. 74. THE OUTPUT Entity E t ti E tit Extraction Issues / Themes / Messages 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 74 @alanchumley / #mra_ac
  75. 75. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES • Words/phrases that  PRE‐SET  are generally good or  are generally good or bad  (Hate.  Love) GENERAL • Caution! 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 75 @alanchumley / #mra_ac
  76. 76. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES • Words/phrases that are  PRE SET  PRE‐SET generally good or bad   generally good or bad (Hate.  Love) GENERAL • Caution! • Financial / market news PRE SET PRE‐SET • Mergers & Acquisitions • Competition SPECIFIC • Better 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 76 @alanchumley / #mra_ac
  77. 77. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES • Words/phrases that are  generally good or bad   generally good or bad PRE‐SET GENERAL (Hate.  Love) • Caution! • Financial / market news PRE‐SET PRE SET • Mergers & Acquisitions SPECIFIC • Competition • Better CUSTOM • Client‐specific context CLIENT/PROJECT • Best SPECIFIC 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 77 @alanchumley / #mra_ac
  78. 78. TAXONOMIES…LEXICONS…DICTIONARIES…BAGS-O-WORDS…CARTRIDGES • Words/phrases that are  generally good or bad   generally good or bad PRE‐SET GENERAL (Hate.  Love) • Caution! • Financial / market news PRE‐SET PRE SET • Mergers & Acquisitions SPECIFIC • Competition • Better CUSTOM • Client‐specific context CLIENT/PROJECT • Best • Multiple cartridges SPECIFIC But! Still needs some human intervention / validation / feedback so it can ‘learn’ and 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 improve carma.com 78 @alanchumley / #mra_ac
  79. 79. AUTOMATED TEXT ANALYTICS VENDORS 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 79 @alanchumley / #mra_ac
  80. 80. 1st GENERATION vs. 2nd GENERATION AUTOMATED SENTIMENT Conceptual Search-Based Key Word (only) Deeper linguistic level p g Based Operates on a semantic level Explicit Understands sentence grammar: References •Subject Counting up the •Predicate •Object “+” & “-” •Who? •Action •Idea •…and the relationships between theseMost studies show that key •…must contain these terms but in a particularword matching is wrong relationship to one anothermore than 50% of the time Opinions, claims, arguments, facts, actions, concepts, messages Non-implicit: theme / spirit / i t t N i li it th i it intent 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 80 @alanchumley / #mra_ac
  81. 81. COMPLICATIONS & PROBLEMS w/ 1ST GEN AUTOMATED APPROACHES •Sentiment may be of interest at multiple levels: •Document •Several Documents •Statement / sentence Statement •Entity •Topic or Theme •Concept •Language is chaotic: •Jargon, slang, (slanguage) •Colloquialism •Forum-specific acronyms •ambiguity, •Anaphora (it, them) Anaphora •Irony, sarcasm •Misspellings •Poor grammar •Variations in expression •Synonyms •Polysemy •Context is key: •Meaning can change depending on context. •Need to look not only at the word but the connective tissues surrounding the word. 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 81 @alanchumley / #mra_ac
  82. 82. CONTEXT IS KEY 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 82 @alanchumley / #mra_ac
  83. 83. ANALYSING THE “CONNECTIVE TISSUE” OF A SENTENCE 1615 M Street, NW; Suite 750 Washington, DC 20036 USA Tel 1.202.842.1818 carma.com 83 @alanchumley / #mra_ac
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×