Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Vovici Vision 2011: Better Surveys Through Social Media


Published on

Published in: Technology, Business
  • Easy and hassle free way to make money online! I have just registered with this site and straight away I was making money! It doesn't get any better than this. Thank you for taking out all the hassle and making money answering surveys as easy as possible even for non-techie guys like me! ▲▲▲
    Are you sure you want to  Yes  No
    Your message goes here
  • Be the first to like this

Vovici Vision 2011: Better Surveys Through Social Media

  1. 1. Better Surveys through Social MediaMay 20, 2011
  2. 2. Gartner’s Hype Cycle 2
  3. 3. MR Firms’ Attitudes towards SMR 63% of MR firms are experimenting with SMR or considering it for the future Only 17% have conducted any social media research to date For 65% of those who are conducting such research, it’s less than 5% of their revenues 66% consider SMR to be neither qual nor quant but something new Source: meaning ltd. 2010 survey 3
  4. 4. Social Media Research at the Peak of Hype Social media research Text analytics MROCs Phone Focus Mail surveys groups surveys Sentiment Mobile analysis research Online access panels Online surveys Research games 4
  5. 5. Social Media Exemplified “I need to pee.” “I peed.” “This is where I pee.” “Why am I peeing?” “Look at this pee!” “I’m good at peeing.” Concept courtesy of 5
  6. 6. Social Links Weak on Twitter; Like Subscriptions Facebook Twitter Researcher Researcher Member with Member Member Member Protected Tweets Updates Updates Member Member often typically inaccessible accessible 6
  7. 7. Survey Research Steps, with Social Media Study Design Use social media to frame hypothesis 7
  8. 8. Market Research in the Groundswell Groundswell: “A social trend in which people use technologies to get the things they need from each other, rather than from traditional institutions like corporations.” Josh Bernoff’s First Rule of Social-Media Market Research “Social applications are typically better at generating hypotheses than at testing them.” “Projecting to the total population is a perilous thing but this is an excellent place to hear what is going on.” 8
  9. 9. Framing HypothesisCase Study: Why are Some Dieters Uninterested in the Atkins Diet? Love of pasta and bread 9
  10. 10. Framing HypothesisCase Study: Why are Some Dieters Uninterested in the Atkins Diet? Love of pasta and bread Distaste for red meat Concerns about heart disease Belief it’s an unproven fad 10
  11. 11. Twitter Search Tips Limited to past 30 days Can constrain geographically Can search on basic sentiments (emoticons) No search counts provided; compare age of oldest tweet for relative measure 11
  12. 12. Tortoise or the Hare: Social Media Sampling Twitter searches are easiest, but only represent 7-8% of online users Keyword searches vs. brand searches Errors of inclusion: Target search returns “likely target”, “target market”, “above target”, etc. Errors of exclusion: McDonalds search excludes “McD”, “Mickey D”, “MickeyDee”, “Golden Arches” Keyword searches vs. constructs “brown” vs. “Charlie Brown”, “Chris Brown”, etc. Source: Conversition 12
  13. 13. Query Bias in Social Media Research Sentiment Varies by Search Term1.000.67 0.330.33 0.27 0.24 0.14 0.090.00 Civ 5 #Civ5 Civilization 5 Civilization V Civ V #CivV-0.33 -0.27-0.67-1.00 13
  14. 14. Formality of Search Terms Affects Type of Results 25% #Civ5 20% Conversational Civ 5 #CivV 15% Civ V 10% Civilization 5 5% Civilization V 0% 0% 20% 40% 60% 80% 100% Newsy 14
  15. 15. Survey Research Steps, with Social Media Sample Selection Identify potential respondents 15
  16. 16. Twitter Sample Sourcing Twitter reaches that part of your audience on Twitter Will differ in key ways from non-Twitter users Qualitative Research More likely to be early adopter of technology Representative More likely to be “Creator” persona of Web usage In fact, probably not representative of your users on Twitter either! 16
  17. 17. Demographics of Twitter Users Source: Quantcast, May 2010 17
  18. 18. Where To Find Them Facebook Fan Pages Twitter Hashtags #MRX #NewMR LinkedIn Discussion Groups 18
  19. 19. Primary Reasons for Returning iPad, for 50 Twitter UsersReturned Considering Upgrade to 3G Poor Value Incompatibilities Other No Reason Given 19
  20. 20. Going Wrong Listening to Customers through Qual “We didnt get the passion this loyal group of consumers have. That wasnt something that came out in the research.” – Tropicana “We have decided to return to our previous terms of use while we resolve the issues that people have raised.” – Facebook “The testing weve done has been incredibly positive.” – Syfy “We have heard your concerns about the ad that was featured on our website.” – Motrin 20
  21. 21. Survey Research Steps, with Social Media Questionnaire Design 1. Learn language of respondents 2. Develop choice lists 21
  22. 22. Need to Learn Language of RespondentsTypical Questionnaires Written from sponsoring organization’s perspective Uses industry jargon Makes subtle distinctionsResult for Respondent Less accurate answers Fewer questions answered Higher abandonment rate 22
  23. 23. MP3 Player SizeSponsor’s Language Customer’s LanguageWhat size of storage do you How many songs do you want to want in an MP3 player? keep on your music player?o 1GB o 500 songso 2GB o 1,000 songso 4GB o 2,000 songso 8GB o 4,000 songso 16GB o 8,000 songs 23
  24. 24. Closed-End Questions Great to Extrapolate From… Top 3 Favorite Colors As Determined by Questions with Different Numbers of Choices15 Except when you get the choice list wrong!10 5 0 A B C D E F Correct Ranking G H 24
  25. 25. Develop Choice Lists for Closed-Ended QuestionsIn the past, why have you thought the Atkins Diet was not right for you? (Choose all that apply.) Love pasta too much to go without Love bread too much to go without Love sweets too much to go without Try to avoid eating red meat Worried about health problems that Atkins might cause Other _________________________________ 25
  26. 26. Survey Research Steps, with Social Media Fielding the Survey Inviting respondents by SM 26
  27. 27. Invite Twitter Users to Your Survey 27
  28. 28. But Recruiting Can be Representative of Behavior… Source: Burke, Inc. 28
  29. 29. Survey Research Steps, with Social Media Analysis Add tweets for color commentary 29
  30. 30. Quantitative Questionnaire Results Lack Detail Representative of U.S. adults who watched the 9-9-09 Obama healthcare address No qualitative insights into why viewers reported this 30
  31. 31. Comments Provide Narrative to the NumbersPositive Negative “President Obama Rocked My World Last “Had to laugh at Obamas speech last Night Greatest Speech Ever by any night! Bi Partiscian negotiations on the President ever in History of USA!” healthcare bill? NOT!!!!!!” “hi good evening! Have u watched “Watching Obama give a speech is like obamas speech on health care? It was a watching someone watch a tennis match good one, as expected. Obama is one as his head just bobs back and from eloquent man.” teleprompter to teleprompter.” “I thought Presidents Obamas speech “Dennis Miller had it right - if POTUS on healthcare was much needed.” cannot start a speech on time, do you trust the govt to deliver healthcare on time & under budget?” 31
  32. 32. “Word Cloud” Highlights Unexpected Theme: Created with Wordle 32
  33. 33. Tweets Must Be Extensively Filtered False News hits Links Shills RTs 25.4% of Spam tweets are replies - HP Conversation Quality Content 33
  34. 34. Tweets as Verbatim ResponsesDifferent types of tweets Practical barriers Shared links Some brands have too little PR feeds public discussion, esp. B2B RTs (Retweets) Some brands are too generic Replies (e.g., Sedona) Small talk Many tweets about public brands are news links 34
  35. 35. Social Media Users as Exceptions They take a public position That position may often be notable as an outlier Position may or may not be true May neither be credible nor reliableSource: Bill Neal, Market Researcher 35
  36. 36. Survey Research Steps, with Social Media Study Design: Use social media to frame hypothesis Sample Selection: Identify potential respondents Questionnaire Design: Learn language of respondents; develop choice lists Fielding the Survey: Invite respondents by Twitter Analysis: Add tweets for color commentary 36
  37. 37. So Long! Thank you for 17.8 great years! Jeffrey Henning @jhenning +1 617-694-4012 37
  38. 38. Appendix
  39. 39. Social Media Defined social media n. information formats produced from activities that integrate technology, social interaction, and the construction of words, pictures, videos, and audio. 39
  40. 40. Research 1.0 vs. Research 2.0 Social Media Research Research 2.0 Focus v MROC Group Research 1.0 Survey Interview Ethnography Directed Moderated Undirected 40
  41. 41. Framing HypothesisCase Study: Why are Some Dieters Uninterested in the Atkins Diet? Love of pasta and bread Distaste for red meat 41
  42. 42. Framing HypothesisCase Study: Why are Some Dieters Uninterested in the Atkins Diet? Love of pasta and bread Distaste for red meat Concerns about heart disease 42
  43. 43. Social Media is a Rich Source of Qualitative Insights Qualitative Research Twittered- Random- Sampled Sample Surveys Representative Research Surveys 43
  44. 44. Most Twitter Users Don’t Participate Much 85 followers, 80 friends 1 tweet a day (255 tweets over 7 months) 68% active in last 30 days Qualitative Research- HP Lab statistics Most sampled tweets therefore literally Representative represent the “vocal minority” 44
  45. 45. Twitter Demographics Source: Dan Zambonini, 45
  46. 46. To Maximize Chance of Being ReTweeted… 46
  47. 47. And Say “Please”! 47
  48. 48. Use Tweets to Trigger Surveys 140 characters of feedback doesn’t provide context Use tweets to trigger survey invitations Best handled by a human rather than automation 5 McDonalds complaints out of 200 tweets 48
  49. 49. Post Questions to LinkedIn 49
  50. 50. Twitter & Social Media One Source Among Many Customer Customer Past Tweets Emails Searches Surveys 50
  51. 51. Copyright Status of Tweets Most Tweets too short and too lacking in “creativity” to qualify for copyright ownership, according to EFF “Short phrases … are not copyrightable, even if such expressions are novel, distinctive, or lend themselves to a play on words.” - Compendium of Copyright Office Practices “When youre talking about 140 characters, youre going into a range where there are probably a lot of things that are copyrightable.” - Jeffrey Johnson, © Attorney, Pryor Cashmen 51
  52. 52. Social Media Research is Not Ethnography Ethnography can involve interviews with singular participants or groups Ethnography can be participatory like mystery shopping Ethnography can be purely observational using video and audio recording Ethnography is about experiencing a social group and their behaviors and attitudes from the insideSource: Chris Bailey 52
  53. 53. Value of Social Media in Survey Research Process Study Design Sample SelectionQuestionnaire Design Fielding the Survey Analysis Not at all Very Sample size: 1 valuable valuable 53
  54. 54. LIRM: Listen, Interpret, React, Monitor Listen. Capture everything. Interpret. Put comments into context and look for Monitor Listen patterns. React. Respond, change and adapt. React Interpret Monitor. Track performance. Source: Bruce Temkin, Experience Matters blog 54
  55. 55. LIRM: Listen, Interpret, React, Monitor “All components of a VoC program need clear mechanisms for capturing everything from customers’ perspective on specific interactions to their satisfaction with the company.” – Bruce Temkin 55
  56. 56. Social Media is Free to Access, Costly to Analyze Yes, you can download recent comments for free However, expect to spend significant amounts of time on data cleaning $ 56
  57. 57. Social Media Research Concerns & Caveats Little content for brands with low market penetration “A good insight comes from triangulation of qualitative and quantitative data to explain the contradictions and inconsistencies between them.” – Ben Foster Artifice vs. Ethnography Social media researcher: “This is a non-interventional way to listen to the authentic Voice of the Customer. Completely unartificial!” Survey researcher: “No artifice is taking a public stance, deciding what to tweet when?! Not natural discourse at all!” 57
  58. 58. Social Media Strike Out Not every brand’s customers are talking about them online Not everything about a brand is talked about, even for those brands people do talk about Not everything talked about has enough context to be useful Not everything talked about is representative of customers 58
  59. 59. J.D. Power’s Blog Research1. Brand monitoring, esp. of competitors2. Trend analysis3. Customer information – cross-post analysis4. Unmet needs – frustrations, wishes, ideas Source: Janet Eden-Harris, VP of Web Intelligence, interviewed by Adam Sultan, Marketing Sherpa 59
  60. 60. LIRM: Listen, Interpret, React, Monitor “Customer feedback needs to be examined by asking questions like ‘Is the issue we’ve uncovered isolated or systemic?’ And ‘Where in our organization can we best deal with this situation?’” – Bruce Temkin 60
  61. 61. Blog & Social Network Content Analysis Gartner sees marginal to no ROI from its clients’ investments in blog content analysis “It’s eye candy: management likes to see it.” Gartner sees no strategic value in connecting customers to their social networks In terms of concrete ROI, instead learn the language of the Voice of the Customer: Ask more open-ended questions in surveys and use text mining on them Use text mining on the keywords customers use in your web site’s search box Source: Gartner, “Analytics to Action: Key Analyses for Customer-Centric Decisions” 61
  62. 62. Building Models with Predictive Validity Weekend box-office receipts – Predicting the Future with Social Media predicted opening box office receipts of U.S. movies with 97.3% accuracy Consumer confidence – From Tweets to Polls found a 73.1% correlation between “job” tweets & Gallup consumer confidence Presidential approval – Found a 72.5% correlation between tweets about Obama and presidential job approval polls Congressional elections – The Daily Beast and WiseWindow built a social media model that predicted 97% of Senate races and 87% of the House races that it tracked 62
  63. 63. Word of Mouth Models Do gains in share of word of mouth – or “word of mouse” – precede gains in actual market share? Build a Twitter tracker of a brand and its major competitors Good social media monitoring tools let you build chronological searches into historic social media data Does your model predict actual gains? Bonus Slide 63
  64. 64. LIRM: Listen, Interpret, React, Monitor “For each component of a VoC program, firms need explicit processes for making changes throughout the organization – based on what is learned from customer insights.” – Bruce Temkin 64
  65. 65. Social Media Monitoring Tools vary by Departmental Need Social Media Brand Service Research Monitoring MonitoringDepartment Market Marketing Customer Research ServiceMission Understand Influence ServeReports Ad Hoc Trends & Priorities CampaignsSegmentation Demographics Influencers Individuals No interaction Extensive interaction 65
  66. 66. LIRM: Listen, Interpret, React, Monitor “As with any well-run corporate program, each component of a VoC program needs automatic feedback loops that track work plans and results.” – Bruce Temkin 66
  67. 67. Monitoring Tools Still Immature Many false hits for keywords, especially generic terms Spam often returned Sentiment analyses only 70% accurate Difficult to screen for content-specific content Influence analyses often use old data and “bogus” algorithms Time consuming Source: Asi Sharabi 67
  68. 68. Types of Online Research CommunitiesClosed OLFG BBFG Insight Community Community Panel Idea Jam Open Idea Voting Temporary Permanent 68
  69. 69. Social Media Research “[Social media] is neither a be-all and end-all nor a fad.” – Ben Smithee, founder, Spych Market Analytics Apply it where appropriate in your current projects Experiment and play! Share the results where possible 69
  70. 70. What are the Rules? Respect/ignore expectations of privacy. Respect privacy: “In a public social media site, they dont want us there. People dont go to social media sites to talk to companies. They go to talk to their friends.” – Jan Trent, Wendy’s It’s not private: “I would say that you should quote anything taken from a blog verbatim and attribute it. Reading content of blogs is desk research amongst publicly published material, not primary research.” - Audrey Anand, Listengroup 70
  71. 71. What are the Rules? Respect/ignore expectations of privacy. Seek/dont seek permission to share consumer comments in your research. “I always ask for the permit from the blogger. Only once or twice I was asked not to use data.” - Mirjana Necak, PR manager “Would it ever be possible to really get informed consent without affecting the quality of the learning, especially if youre asking over the net?” - Letesia Gibson, State of Play 71
  72. 72. What are the Rules? Respect/ignore expectations of privacy. Seek/dont seek permission to share consumer comments in your research. Cite/obscure identities of commenters. “I would take all the comments I wanted (maybe change word order a bit) and then apply usual criteria to hide contact info. I would offer categories of participants.” - Kathy Flament, Flament Associates “To mask its source and treat it as if it was from a ‘live respondent’ would be a kind of plagiarism of the original author and to some degree dishonest to any client reading it.” - Audrey Anand 72
  73. 73. What are the Rules? …Cite/obscure identities of commenters. Engage/dont engage with commenters. Bloggers merit a reciprocal relationship. Engage & connect in order to understand context of what is being shared, meeting bloggers half way. - paraphrase of Josephine Hansom, with GfK NOP “In the course of carrying out social media research, someone replies to someone whose data just happens to appears in the research data set. The person didnt ask to participate and they didnt respond to a question. For me, this is in direct violation of the Prime Directive.” - Annie Pettit, Conversition 73
  74. 74. What are the Rules? Respect/ignore expectations of privacy. Seek/dont seek permission to share consumer comments in your research. Cite/obscure identities of commenters. Engage/dont engage with commenters. 74
  75. 75. What do Researchers Think? Respect (60%)/ignore (26%) expectations of privacy Cite (26%)/obscure (53%) identities of commenters Seek (50%)/dont seek (33%) permission Engage (59%)/dont engage (24%) with commenters Source: 226 attendees to AMA webinar, "Excuse Me! Were Having a Conversation Here!" 75
  76. 76. Look at Ourselves in the Mirror 76
  77. 77. What do Consumers Think? 77
  78. 78. SurveyMethodology Limitations Online survey of 426 U.S. Convenience sample panel members of Western Instrument bias Wats People willing to answer surveys may have a more May, 2010 positive view of MR No weighting Facebook privacy concerns may have heightened respondent concerns 78
  79. 79. AAPOR Standard Disclosure Form BASIC DISCLOSURE ELEMENTS DETAILS Survey sponsor Vovici Data collection supplier Western Wats Opinion Outpost Population represented Online adults in the United States Sample size 426 Mode of data collection Web survey Type of sample Non-probability Start/end dates of data collection May 7, 2010 to May 10, 2010 Margin of sampling error Not applicable Are the data weighted? No Response rate (and how calculated) 8.5% (AAPOR RR2) Contact for question text and more Jeffrey Henning information 79
  80. 80. What Do Customers Like About Social Media Research? “It provides marketing researchers with an understanding of the real life reaction to a given product. That, in turn, leads to a better product for consumers.” “They can provide a better product/service based on what the actual customers are saying, rather than the focus group they put together.” “It makes sense. It would be the most raw and most likely emotionally real response that could be given.” “I like that when studying comments, those comments are actually put into action to please their customers.” 80
  81. 81. What Do Consumers Dislike? “A little too close to ‘Big Brother is watching’. But you should know enough to never post if you dont want others to read it. 81
  82. 82. What Do Consumers Dislike? “A little too close to ‘Big Brother is watching’. But you should know enough to never post if you dont want others to read it.” “I dont like that they invade our privacy in the first place. If we wanted them to know, we would contact them.” “I like nothing about this. If it is social media, it should be social media and not for research.” “It is skeezy and lecherous stealing data that they would otherwise have to pay for.” “I feel that their opinion is a waste of time and that this type of job should be eliminated.” 82
  83. 83. Concern for Privacy How concerned are you about your privacy on the Internet?100% 90% 80% 70% 60% 50% 40% 33% 30% 23% 24% 20% 16% 10% 5% 0% Not at all Slightly Moderately Very Extremely concerned concerned concerned concerned concerned n = 426 83
  84. 84. Concerned for Privacy, Really?35% 23% 33% 24% 16% 5%-15% Not at all concerned Slightly concerned Moderately concerned Very concerned Extremely concerned People concerned about Internet privacy dont engage in fewer online activities. Concerned users dont vary in which social networks they use or how many social networks they use. Concerned users dont comment less often on web sites than unconcerned users do. Concerned users arent less likely to use photos of themselves or their real name or email address when posting comments. n = 426 84
  85. 85. “Um, We Didn’t Know You Were Listening” As far as you know, which types of organizations monitor and analyze public Internet discussions? "Some organizations" 69%Law enforcement agencies 52% Market researchers 45% Social networks 42% Search engines 37% Marketing departments 35% Online stores 25% Service departments 15% 0% 20% 40% 60% 80% 100% n = 426 85
  86. 86. Framing the Discussion“Market researchers, service departments and marketingdepartments often analyze comments made on the Internet tounderstand consumer attitudes and satisfaction with products andservices. Researchers want to understand what people think; servicedepartments want to follow up to improve satisfaction; marketingdepartments may want to make you aware of competing products.Assume you make a comment about a product or service on theInternet…” Social Media Service Brand Market Monitoring Monitoring Research 86
  87. 87. Privacy “Can violate privacy if we didnt know it could happen.” “It interferes with my understanding of privacy.” “When they can identify me, follow my activities about everything I do on the internet and target me specifically, then its gets a little scary and I worry about my privacy.” “Make sure to keep privacy guidelines to those who dont want you to use their words.” “It often times invades the privacy of the individuals they are tracking.” 87
  88. 88. Perceptions of Privacy: Depends on your AnalogyMarket Researchers Social media conversations are secondary research Matter of public record 88
  89. 89. Perceptions of Privacy: Depends on your Analogy Social media conversations are conversations…stop eavesdropping! Conversations in public places aren’t public Researchers aren’t the intended audience 89
  90. 90. Sharing Consumer Comments Would you prefer researchers…?Ask for permission before reporting your comments 85% “They sometimes take things out of context and should always contact the person before using their Use your comments without contacting you 4% comments.” “The idea that comments that I make could be taken out of context to support a product or position Don’t care 11% that I dont personally support.” 0% 20% 40% 60% 80% 100% n = 419 90
  91. 91. Identity when Commenting “Screen name” or other invented name 70% Email address 45% Real name 40% Picture 23% How they Anonymously identify 22% themselves Photo portrait 15% Geolocation 5% None of the above 1%Identify you by your real name (if given) 7% Identify you by your screen name (if… 20% How they want Describe you by your demographics 24% researchers to Not identify you at all 43% identify them Don’t care 7% 0% 20% 40% 60% 80% 100% n = 419 91
  92. 92. Contact is Conditional…On Who Comments Where Which Groups Is It Acceptable to Contact You The organization you comment on 59% Company representatives who want to address your comment 40% Market researchers with the organization you comment on 30% Other members of the community or site you posted to 25%Marketers who want to make you aware of a competing product 18% Competitors to the organization you comment on 17% Independent market researchers 15% None of the above 23% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% n = 422 92
  93. 93. Contact is Conditional…On Who Comments Where Which Sites May Groups Contact You Through (Base = Users of Such Sites) An online store 33% Your personal Twitter page 32% Your online journal or blog 32% Your personal MySpace page 22% Your personal LinkedIn page 21%An online news group, website, blog or photo site 21% A social networking site 21% Your personal Facebook page 19% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% n = 422 , 66 , 63 , 89 , 42 , 422 , 284 , 277 93
  94. 94. Consumer-Driven Rules of Social Media Market Research Respect expectations of privacy. Seek permission to share consumer comments in your research. Obscure identities of commenters. Dont engage with commenters. 94
  95. 95. Vovici CustomersFinancial Services “We have 14,000 surveys going out at all times, and the ability to personalize them has been huge for our response rates which reach up to 80% completion rate.”IT / Telecommunications Healthcare Robert Kimmel, Leadership Development CenterTransportation Manufacturing “Vovici is the solution Oracle counts on to build a world- class VOC program. Since using Vovici, we improved our response rates by 25% resulting in over 200,000 responses this year alone.” Jeremy Whyte, Dir. Customer Feedback & Reporting, OracleEducational / Government Government of Canada “Vovici’s real-time alerts and email triggers allow us to run our business around customer responsiveness. Now itConsumer / Retail takes an employee a half a day to capture the insights it would have taken us 6 months to get.” Marsha Jones, Manager, Lead Acquisition and Solutions 95