Distilling Actionable                                                      Insights from the Deluge of                    ...
From Data Deluge to Insights                               © 2012 Converseon Inc. Proprietary and Confidential   2
The vast scope of social media data available todayrequires scalable tech solutions. Human-machinecollaboration is the onl...
Social-media research can support both traditionalmarket research goals and PR use cases.Traditional Market Research      ...
These two use cases – market research andcommunications – closely align with two services.Social Listening                ...
The Social Media Research Process: From Raw Data toInsights               1. Data             2. Data              Collect...
Stage 1: Social Data Collection                                              Primary Goal:                                ...
Stage 2: Data Enrichment                                                  Primary Goal:                                   ...
Stage 3: Analysis & Insight Generation                                           Primary Goal:                            ...
Social media is a massive compendium of documents…                                 © 2012 Converseon Inc. Proprietary and ...
Harvesting Data and Metadata from Social MediaDocuments: A Tweet Dissected                                   © 2012 Conver...
Harvesting Data and Metadata from Social MediaDocuments: A Tweet DissectedDatapoints:• Author Name• Text• Publication Date...
Harvesting Data and Metadata from Social MediaDocuments: A Tweet DissectedMetadata:• Person or tweet that a  tweet is in r...
Sorting Social Metadata                          A                                                       B                ...
Relevancy as a Sorting Task…                                                    Irrelevant Documents All Social Media Docu...
Data Enrichment: What Should We Measure?   Metric                  Explanation   Sentiment               Does the author m...
Data Enrichment: What Should We Measure?   Metric                  Sorting Categories   Sentiment               Positive, ...
How can we implement the sorting tasks we’vediscussed so far?Machine Sorters                               Human Sorters  ...
Q: How do you know when a computer is correct?    A: The same way you know that a human is correct:     “I know it when I ...
Establishing A Basis for How Well Humans Agree WithOne AnotherExample 1: Inter-Coder Agreement on Sentiment   Example 2: I...
Using Human Parallel Coding to Establish GoldStandards              Confusion Matrix: Human as Gold Standard              ...
Using A Credit Matrix to Create Improved Measurement                        Credit Matrix             POSITIVE     NEGATIV...
But how does the machine learn? 1. Collection of Human   2. Machine ingests coded data     3. Machine applies model from  ...
In conclusion….                  © 2012 Converseon Inc. Proprietary and Confidential   24
Thank You!Jasper Snyder,VP, Converseonjsnyder@converseon.com                                                              ...
Upcoming SlideShare
Loading in …5
×

Converseon 2012 CASRO Technology Conference

1,449 views

Published on

Presentation given by Jasper Snyder of Converseon to 2012 CASRO Technology Conference, 6/30/12.

Published in: Business, Technology
  • Be the first to comment

Converseon 2012 CASRO Technology Conference

  1. 1. Distilling Actionable Insights from the Deluge of Social Media Data Jasper Snyder VP, Converseon© 2012 Converseon Inc. Proprietary and Confidential
  2. 2. From Data Deluge to Insights © 2012 Converseon Inc. Proprietary and Confidential 2
  3. 3. The vast scope of social media data available todayrequires scalable tech solutions. Human-machinecollaboration is the only way to deal with this deluge. Social Media Channel Approx. Monthly Volume Furthermore… On-site comments and Blogs 30 million new posts social cues and sharing Social cues (e.g., “likes”) Facebook 1.8 billion status updates and comments Social cues like favoriting Twitter 4 billion tweets and flagging other users 240 years of video YouTube 400 million social actions content uploaded each month © 2012 Converseon Inc. Proprietary and Confidential 3
  4. 4. Social-media research can support both traditionalmarket research goals and PR use cases.Traditional Market Research Communications Functions through Socialthrough Social Media Listening Media Monitoring • Consumer • Consumer complaints and Segmentation product malfunctions • Purchase triggers • Adverse reactions for pharmaceutical companies • Thoughts and opinions about products and • Crisis monitoring and brands response • Market awareness of • Reputation management products or brands © 2012 Converseon Inc. Proprietary and Confidential 4
  5. 5. These two use cases – market research andcommunications – closely align with two services.Social Listening Social Media MonitoringWhen what matters most is When what matters most is deliveringunderstanding a consumer segment or customer service, navigating a crisismarket. situation or detecting reputation threats.Goal is to acquire just enough data to Goal is comprehensive, real timeunderstand a population “out there” in coverage.the world.Higher tolerance for missing content. Higher tolerance for irrelevant content.Lower tolerance for irrelevant content. Lower tolerance for missing content. © 2012 Converseon Inc. Proprietary and Confidential 5
  6. 6. The Social Media Research Process: From Raw Data toInsights 1. Data 2. Data Collection Enrichment 3. Analysis & Insight Generation © 2012 Converseon Inc. Proprietary and Confidential 6
  7. 7. Stage 1: Social Data Collection Primary Goal: Identify and acquire the data 1. Data 2. Data that can answer your business Collection Enrichment questions. Primary Challenges: 1. Pull in relevant data and 3. Analysis & metadata Insight Generation 2. Coverage of appropriate social media channels 3. Eliminate spam and irrelevant content. © 2012 Converseon Inc. Proprietary and Confidential 7
  8. 8. Stage 2: Data Enrichment Primary Goal: 2. Data Implement document- and sub-1. Data Collection Enrichment document-level enrichments like topic, consumer segment, emotion and sentiment. Primary Challenges: 3. Analysis & 1. Data normalization Insight Generation 2. Classification 3. Scalability © 2012 Converseon Inc. Proprietary and Confidential 8
  9. 9. Stage 3: Analysis & Insight Generation Primary Goal: 2. Data Connect the dots between a1. Data Collection Enrichment suite of metrics and data points in order to reach sound strategic conclusions. Primary Challenges: 3. Analysis 1. Reliability & Insight Generation 2. Strategic Value © 2012 Converseon Inc. Proprietary and Confidential 9
  10. 10. Social media is a massive compendium of documents… © 2012 Converseon Inc. Proprietary and Confidential 10
  11. 11. Harvesting Data and Metadata from Social MediaDocuments: A Tweet Dissected © 2012 Converseon Inc. Proprietary and Confidential 11
  12. 12. Harvesting Data and Metadata from Social MediaDocuments: A Tweet DissectedDatapoints:• Author Name• Text• Publication Date• Some hashtags © 2012 Converseon Inc. Proprietary and Confidential 12
  13. 13. Harvesting Data and Metadata from Social MediaDocuments: A Tweet DissectedMetadata:• Person or tweet that a tweet is in reply to• Follower count of author• Times retweeted• Times favorited• Author description © 2012 Converseon Inc. Proprietary and Confidential 13
  14. 14. Sorting Social Metadata A B C Tweets that contain #Ford in the text. © 2012 Converseon Inc. Proprietary and Confidential 14
  15. 15. Relevancy as a Sorting Task… Irrelevant Documents All Social Media Documents • Spam • Documents not in target All Documents language (e.g., not English) Containing Your Boolean Query • Contain keyword but not relevant to client question Relevant Documents © 2012 Converseon Inc. Proprietary and Confidential 15
  16. 16. Data Enrichment: What Should We Measure? Metric Explanation Sentiment Does the author make a negative or positive point about a product or brand? Topics What topic is the author talking about the product or brand in relation to? Purchase Stage Has the author of a document already purchased the product when writing about it online? Consumer Segmentation What segment is the document’s author from? Emotions What emotions do authors express toward the target brand or product? © 2012 Converseon Inc. Proprietary and Confidential 16
  17. 17. Data Enrichment: What Should We Measure? Metric Sorting Categories Sentiment Positive, negative, neutral Topics Pre-selected topic and unexpected topics Purchase Stage Before making a purchase or after. Consumer Segmentation Young male, middle-aged woman, etc. Emotions Joy, anticipation, surprise, fear, etc. © 2012 Converseon Inc. Proprietary and Confidential 17
  18. 18. How can we implement the sorting tasks we’vediscussed so far?Machine Sorters Human Sorters Sorting Tasks © 2012 Converseon Inc. Proprietary and Confidential 18
  19. 19. Q: How do you know when a computer is correct? A: The same way you know that a human is correct: “I know it when I see it…” © 2012 Converseon Inc. Proprietary and Confidential 19
  20. 20. Establishing A Basis for How Well Humans Agree WithOne AnotherExample 1: Inter-Coder Agreement on Sentiment Example 2: Inter-Coder Agreement on Emotion Item Coder 1 Coder 2 Tweet Coder 1 Coder 2 I do not like the cats with Disgust Anger 1 Positive Positive thumbs “advert” 2 Positive Neutral I say that video is real, Trust No Emotion definitely. Expressed 3 Neutral Neutral 4 Negative Positive etc. … … © 2012 Converseon Inc. Proprietary and Confidential 20
  21. 21. Using Human Parallel Coding to Establish GoldStandards Confusion Matrix: Human as Gold Standard POSITIVE NEGATIVE NEUTRAL TOTAL POSITIVE 365 24 159 548 NEGATIVE 57 81 65 203 Raw Accuracy: 61.5% NEUTRAL 274 60 415 749 TOTAL 696 165 639 1500 © 2012 Converseon Inc. Proprietary and Confidential 21
  22. 22. Using A Credit Matrix to Create Improved Measurement Credit Matrix POSITIVE NEGATIVE NEUTRAL POSITIVE 100% 0% 50% NEGATIVE 0% 100% 50% NEUTRAL 50% 50% 100% Partial Credit Figure of Merit: 82.3% Confusion Matrix: Human 1 as Gold Standard POSITIVE NEGATIVE NEUTRAL POSITIVE 365 24 159 NEGATIVE 57 81 65 NEUTRAL 274 60 415 © 2012 Converseon Inc. Proprietary and Confidential 22
  23. 23. But how does the machine learn? 1. Collection of Human 2. Machine ingests coded data 3. Machine applies model from and finds patterns in each Annotated Data category classification step two on raw data. Results are compared to human coding of same material. © 2012 Converseon Inc. Proprietary and Confidential 23
  24. 24. In conclusion…. © 2012 Converseon Inc. Proprietary and Confidential 24
  25. 25. Thank You!Jasper Snyder,VP, Converseonjsnyder@converseon.com Converseon Inc. 53 West 36th Street, 8th Floor, New York, NY 10018 t: 212.213.4279 | f: 646.304.2364 www.converseon.com 25 © 2012 Converseon Inc. Proprietary and Confidential

×