Biting the hand andbending the rulesBy Annie Pettit PhD, Chief Research Officer, ConversitionBestsentimentsystemever!
Our sentimentis99% validMisinformation #14
Real-life Data Analysis: How Many Licks to the Tootsie Roll Center of a Tootsie Pop?by Cory Heid, guest bloggerAlmost all ...
• I want Cracker Jacks.• Cracker Jacks are the shit• i want Cracker Jacks soobad• I am addicted to CrackerJacks...Short ve...
Which sentiment trend is more valid?75=Positive1=Negative3.13.23.33.4Short Long
“The best academic study Ive seen showed only82% agreement between two human sentimentannotators, rising to 90% if uncerta...
Sentimentscoring is onesize fits allMisinformation #29
One size scoring or custom scoring?1010%20%30%40%50%60%Jan Feb Mar Apr May Jun Jul Aug SepOneSize Scoring Custom Scoring
Standard scoring or category scoring?1120%40%60%80%Standard Category
Twitter(survey)is a GodsendMisinformation #312
0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutra...
I can measureyour brandawarenessMisinformation #414
Pfizer has greater awareness than Unilever150100002000030000Pfizer Unilever
P90X has even greater awareness than Pfizer16020000400006000080000P90X Pfizer Unilever Red Rose TeaCracker Jacksand Tootsi...
Demographicsare availableMisinformation #517
Care to Crosstab Demos on Starbucks Data?18Total MaleFemaleUSEuropeAsiaCanadaMiddle EastPacific Africa 25-34 18-24 35-44 4...
Social mediaresults areinstantaneousMisinformation #619
Antacid: Fast or Good?200%20%40%60%Negative Neutral Positive0%20%40%60%80%Negative Neutral PositiveFast: With Spam Good: W...
What is GMC?21
GMC: Fast or Good?220%25%50%75%100%GMC Auto GMC AnyB2B N T2B
Insight isinstantMisinformation #723
Expertise is required24+ =
The internetwas made forengagementMisinformation #825
Famous people signed up for this26
27
28
29
• http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf• http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines....
Who’s a member?31
Why aren’t listening companies members?32
Stifles my creativity#1 Why not?33
I don’t want someone tellingme what to do#2 Why not?34
They will put me out ofbusiness#3 Why not?35
Confessions• Strengths and weaknesses• Sentiment analysis isnowhere near perfect• Speed will not result inhigh quality• De...
Thank you!37
"Biting the hand and bending the rules"
"Biting the hand and bending the rules"
Upcoming SlideShare
Loading in …5
×

"Biting the hand and bending the rules"

423 views

Published on

Chicago AMA Research SIG event on May 20th. Presentation delivered by Annie Pettit, PhD., Chief Research Officer, Conversition

Published in: Education, Technology, Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
423
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
3
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

"Biting the hand and bending the rules"

  1. 1. Biting the hand andbending the rulesBy Annie Pettit PhD, Chief Research Officer, ConversitionBestsentimentsystemever!
  2. 2. Our sentimentis99% validMisinformation #14
  3. 3. Real-life Data Analysis: How Many Licks to the Tootsie Roll Center of a Tootsie Pop?by Cory Heid, guest bloggerAlmost all of us have tried a Tootsie Pop at some point. I’m willing to bet that most of us also thought, “I wonderhow many licks it does take to get to the center of the Tootsie Pop?” If you haven’t wondered about this, here’sthe classic commercial that may get you more curious.Personally, I was not very satisfied with the owls answer of “3,” so I decided to continue the little boy’s quest tofind the number of licks required to reach the center of a Tootsie Pop.ResearchLooking around the ‘net, I found that other studies done by student researchers at various universities havereached very different answers. These students, who represented some significant research institutions andengineering schools, used different licking methodologies and/or licking machines and licking experiments. As Imentioned, the results varied greatly. See for yourself:Since I lacked both the equipment and desire to build a licking machine, a simpler licking experiment seemedappropriate. I wanted to assess which of these factors would have the most impact on the effectiveness of theTootsie-Pop licking:Force of the lickTemperature of the lickers mouthpH Level of the lickers salivaSolubility of the lickers salivaLab WorkOnce I selected my factors, I incorporated them into an experiment. I chose simulate the different levels of eachfactor in the lab. For my force simulation, I placed my Tootsie Pop in a beaker with 150 mL of water and amagnetic stirrer. I placed the beaker on a hot plate that could spin the stirrer and create a circular motion in thewater. I then pulled the Tootsie Pop out of the water every minute and measured the height and width of thepop on the thicker band around the pop. I did this each minute until the pop revealed a noticeable amount ofchocolate (the elusive ‘center’).This method had some issues. The chocolate-flavored Tootsie Pops made the water very murky, so I omittedthem from the lab testing. I also could not track the speed of the circulating water. Instead, I used the speeddial’s indicator on the magnetic stirrer (1 being slowest speed, 10 being fastest speed). After several tests atspeeds 1, 2, 4 and 6, I was able to gather an ample amount of data to perform some analysis.I then moved on to the temperature tests. Like the force tests, these used a beaker, 150 mL of water, and a hotplate. I heated the water to different temperatures and measured the height and width of the Tootsie Pop everyLong Verbatims have poor validity5
  4. 4. • I want Cracker Jacks.• Cracker Jacks are the shit• i want Cracker Jacks soobad• I am addicted to CrackerJacks...Short verbatims have high validity6
  5. 5. Which sentiment trend is more valid?75=Positive1=Negative3.13.23.33.4Short Long
  6. 6. “The best academic study Ive seen showed only82% agreement between two human sentimentannotators, rising to 90% if uncertain cases areremoved.“Seth Grimes, June 2011,http://www.b-eye-network.com/view/15276The Truth8
  7. 7. Sentimentscoring is onesize fits allMisinformation #29
  8. 8. One size scoring or custom scoring?1010%20%30%40%50%60%Jan Feb Mar Apr May Jun Jul Aug SepOneSize Scoring Custom Scoring
  9. 9. Standard scoring or category scoring?1120%40%60%80%Standard Category
  10. 10. Twitter(survey)is a GodsendMisinformation #312
  11. 11. 0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 Boxes0%20%40%60%Bottom 2 Boxes Neutral Top 2 BoxesGuess the brands (Tootsie Roll, Chicago Bears,Cracker Jack, Ferris Wheels)13BlogspotTumblrTwitterFacebook
  12. 12. I can measureyour brandawarenessMisinformation #414
  13. 13. Pfizer has greater awareness than Unilever150100002000030000Pfizer Unilever
  14. 14. P90X has even greater awareness than Pfizer16020000400006000080000P90X Pfizer Unilever Red Rose TeaCracker Jacksand Tootsie Rollsare here
  15. 15. Demographicsare availableMisinformation #517
  16. 16. Care to Crosstab Demos on Starbucks Data?18Total MaleFemaleUSEuropeAsiaCanadaMiddle EastPacific Africa 25-34 18-24 35-44 45-54 55+Sample Size 18557 4182 3706 2118 358 247 220 47 35 20 12 9 6 6 205000100001500020000
  17. 17. Social mediaresults areinstantaneousMisinformation #619
  18. 18. Antacid: Fast or Good?200%20%40%60%Negative Neutral Positive0%20%40%60%80%Negative Neutral PositiveFast: With Spam Good: Without Spam
  19. 19. What is GMC?21
  20. 20. GMC: Fast or Good?220%25%50%75%100%GMC Auto GMC AnyB2B N T2B
  21. 21. Insight isinstantMisinformation #723
  22. 22. Expertise is required24+ =
  23. 23. The internetwas made forengagementMisinformation #825
  24. 24. Famous people signed up for this26
  25. 25. 27
  26. 26. 28
  27. 27. 29
  28. 28. • http://www.mra-net.org/rq/documents/MRA_IMRO_SMR16.pdf• http://www.casro.org/pdfs/1011/Social_Media_Research_Guidelines.pdf• http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-on-Social-Media.pdf• http://www.mrs.org.uk/pdf/2012-04-04%20Online%20data%20collection%20and%20privacy.pdfMRX guidelines say don’t do that
  29. 29. Who’s a member?31
  30. 30. Why aren’t listening companies members?32
  31. 31. Stifles my creativity#1 Why not?33
  32. 32. I don’t want someone tellingme what to do#2 Why not?34
  33. 33. They will put me out ofbusiness#3 Why not?35
  34. 34. Confessions• Strengths and weaknesses• Sentiment analysis isnowhere near perfect• Speed will not result inhigh quality• Demographics are weak• Stance on privacyWhat to look for36
  35. 35. Thank you!37

×