"Biting the hand and bending the rules"
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


"Biting the hand and bending the rules"



Chicago AMA Research SIG event on May 20th. Presentation delivered by Annie Pettit, PhD., Chief Research Officer, Conversition

Chicago AMA Research SIG event on May 20th. Presentation delivered by Annie Pettit, PhD., Chief Research Officer, Conversition



Total Views
Views on SlideShare
Embed Views



1 Embed 1

https://twitter.com 1


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

"Biting the hand and bending the rules" Presentation Transcript

  • 1. Biting the hand andbending the rulesBy Annie Pettit PhD, Chief Research Officer, ConversitionBestsentimentsystemever!
  • 2. Our sentimentis99% validMisinformation #14
  • 3. Real-life Data Analysis: How Many Licks to the Tootsie Roll Center of a Tootsie Pop?by Cory Heid, guest bloggerAlmost all of us have tried a Tootsie Pop at some point. I’m willing to bet that most of us also thought, “I wonderhow many licks it does take to get to the center of the Tootsie Pop?” If you haven’t wondered about this, here’sthe classic commercial that may get you more curious.Personally, I was not very satisfied with the owls answer of “3,” so I decided to continue the little boy’s quest tofind the number of licks required to reach the center of a Tootsie Pop.ResearchLooking around the ‘net, I found that other studies done by student researchers at various universities havereached very different answers. These students, who represented some significant research institutions andengineering schools, used different licking methodologies and/or licking machines and licking experiments. As Imentioned, the results varied greatly. See for yourself:Since I lacked both the equipment and desire to build a licking machine, a simpler licking experiment seemedappropriate. I wanted to assess which of these factors would have the most impact on the effectiveness of theTootsie-Pop licking:Force of the lickTemperature of the lickers mouthpH Level of the lickers salivaSolubility of the lickers salivaLab WorkOnce I selected my factors, I incorporated them into an experiment. I chose simulate the different levels of eachfactor in the lab. For my force simulation, I placed my Tootsie Pop in a beaker with 150 mL of water and amagnetic stirrer. I placed the beaker on a hot plate that could spin the stirrer and create a circular motion in thewater. I then pulled the Tootsie Pop out of the water every minute and measured the height and width of thepop on the thicker band around the pop. I did this each minute until the pop revealed a noticeable amount ofchocolate (the elusive ‘center’).This method had some issues. The chocolate-flavored Tootsie Pops made the water very murky, so I omittedthem from the lab testing. I also could not track the speed of the circulating water. Instead, I used the speeddial’s indicator on the magnetic stirrer (1 being slowest speed, 10 being fastest speed). After several tests atspeeds 1, 2, 4 and 6, I was able to gather an ample amount of data to perform some analysis.I then moved on to the temperature tests. Like the force tests, these used a beaker, 150 mL of water, and a hotplate. I heated the water to different temperatures and measured the height and width of the Tootsie Pop everyLong Verbatims have poor validity5
  • 4. • I want Cracker Jacks.• Cracker Jacks are the shit• i want Cracker Jacks soobad• I am addicted to CrackerJacks...Short verbatims have high validity6
  • 5. Which sentiment trend is more valid?75=Positive1=Negative3. Long
  • 6. “The best academic study Ive seen showed only82% agreement between two human sentimentannotators, rising to 90% if uncertain cases areremoved.“Seth Grimes, June 2011,http://www.b-eye-network.com/view/15276The Truth8
  • 7. Sentimentscoring is onesize fits allMisinformation #29
  • 8. One size scoring or custom scoring?1010%20%30%40%50%60%Jan Feb Mar Apr May Jun Jul Aug SepOneSize Scoring Custom Scoring
  • 9. Standard scoring or category scoring?1120%40%60%80%Standard Category
  • 10. Twitter(survey)is a GodsendMisinformation #312
  • 11. 0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 BoxesGuess the brands (Tootsie Roll, Chicago Bears,Cracker Jack, Ferris Wheels)13BlogspotTumblrTwitterFacebook
  • 12. I can measureyour brandawarenessMisinformation #414
  • 13. Pfizer has greater awareness than Unilever150100002000030000Pfizer Unilever
  • 14. P90X has even greater awareness than Pfizer16020000400006000080000P90X Pfizer Unilever Red Rose TeaCracker Jacksand Tootsie Rollsare here
  • 15. Demographicsare availableMisinformation #517
  • 16. Care to Crosstab Demos on Starbucks Data?18Total MaleFemaleUSEuropeAsiaCanadaMiddle EastPacific Africa 25-34 18-24 35-44 45-54 55+Sample Size 18557 4182 3706 2118 358 247 220 47 35 20 12 9 6 6 205000100001500020000
  • 17. Social mediaresults areinstantaneousMisinformation #619
  • 18. Antacid: Fast or Good?200%20%40%60%Negative Neutral Positive0%20%40%60%80%Negative Neutral PositiveFast: With Spam Good: Without Spam
  • 19. What is GMC?21
  • 20. GMC: Fast or Good?220%25%50%75%100%GMC Auto GMC AnyB2B N T2B
  • 21. Insight isinstantMisinformation #723
  • 22. Expertise is required24+ =
  • 23. The internetwas made forengagementMisinformation #825
  • 24. Famous people signed up for this26
  • 25. 27
  • 26. 28
  • 27. 29
  • 28. • http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf• http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf• http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-on-Social-Media.pdf• http://www.mrs.org.uk/pdf/2012-04-04%20Online%20data%20collection%20and%20privacy.pdfMRX guidelines say don’t do that
  • 29. Who’s a member?31
  • 30. Why aren’t listening companies members?32
  • 31. Stifles my creativity#1 Why not?33
  • 32. I don’t want someone tellingme what to do#2 Why not?34
  • 33. They will put me out ofbusiness#3 Why not?35
  • 34. Confessions• Strengths and weaknesses• Sentiment analysis isnowhere near perfect• Speed will not result inhigh quality• Demographics are weak• Stance on privacyWhat to look for36
  • 35. Thank you!37