Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Fear and loathing on the social campaign trail

54 views

Published on

Slides from a talk I give now and then on the history of fear and loathing on the campaign trail.

Published in: Social Media
  • Be the first to comment

  • Be the first to like this

Fear and loathing on the social campaign trail

  1. 1. Fear and Loathing on the Social Campaign Trail Dr. Stuart Shulman @stuartwshulman
  2. 2. Part One Political Fear and Loathing in United States History
  3. 3. “The distinguishing thing about the paranoid style is not that its exponents see conspiracies or plots here and there in history, but that they regard a vast' or gigantic' conspiracy as the "motive force" in historical events...The paranoid spokesman sees the fate of this conspiracy in apocalyptic terms--he traffics in the birth and death of whole worlds, whole political orders, whole systems of human values. He is always manning the barricades of civilization.”
  4. 4. Emergent properties found in a very well read texts, such as the character type “extremist agent of the law”
  5. 5. Part Two The Social Campaign Trail
  6. 6. Part Three Previous Work on Methods & Tools
  7. 7. The NRC emotion lexicon is a list of words and their associations with eight emotions (anger, fear, anticipation, trust, surprise, sadness, joy, and disgust) and two sentiments (negative and positive).
  8. 8. The First 150 of 1476 NRC Fear Words
  9. 9. Coding Fear Words
  10. 10. Coding Fear Words
  11. 11. - MIT Professor Eric von Hippel “This is really the biggest paradigm shift in innovation since the Industrial Revolution” Crowdsourcing brings widely distributed wisdom to process of text analysis
  12. 12. Coder Count Unit Count 13 9 12 47 11 27 10 80 9 155 8 67 7 27 6 8 5 17 4 8 3 474 2 957 1 1757 Crowds Create New Filtering Options
  13. 13. Measurement of Coder Agreement
  14. 14. Defined Search - Incompetence
  15. 15. Clearly Fear?
  16. 16. Adjudicating Fear Coding
  17. 17. Clearly Fear?
  18. 18. Ideology
  19. 19. Custom Classifier Histogram
  20. 20. Filtering Using a Classifier Histogram
  21. 21. Part Four New Work on Methods & Tools
  22. 22. First Person Political Fear Tweets
  23. 23. Version 1 Gnip PowerTrack Rule i (hate OR fear OR loathe OR despise OR dislike OR abhor OR aversion OR afraid OR scared OR dismay OR dread OR horror OR alarm OR frightened OR frightful OR horrified OR terrified) "american politics" lang:en -is:retweet First Person Tweet Collection
  24. 24. Version 2 Gnip PowerTrack Rule (hate OR fear OR loathe OR despise OR dislike OR abhor OR aversion OR afraid OR scared OR dismay OR dread OR horror OR alarm OR frightened OR frightful OR horrified OR terrified) "american politics" lang:en -is:retweet First Person Tweet Collection
  25. 25. First Person Tweet Collection Version 3 Gnip PowerTrack Rule ("i fear" OR "i am afraid" OR "i'm scared" OR "i am scared" OR "i am worried" OR "i'm worried" OR "i dread" OR "i am horrified" OR "i worry" OR "i feel afraid" OR "i feel scared" OR "i am terrified" OR "i feel terrified" OR "i feel worried" OR "worries me" OR "scares me" OR "frightens me" OR "horrifies me" OR "terrifies me") (trump) lang:en -is:retweet
  26. 26. Version 4 Gnip PowerTrack Rule ("i fear" OR "i am afraid" OR "i'm scared" OR "i am scared" OR "i am worried" OR "i'm worried" OR "i dread" OR "i am horrified" OR "i worry" OR "i feel afraid" OR "i feel scared" OR "i am terrified" OR "i feel terrified" OR "i feel worried" OR "worries me" OR "scares me" OR "frightens me" OR "horrifies me" OR "terrifies me") (libtard OR democrat OR liberal) lang:en -is:retweet First Person Tweet Collection
  27. 27. Version 5 Gnip PowerTrack Rule ("i fear" OR "i am afraid" OR "i'm scared" OR "i am scared" OR "i am worried" OR "i'm worried" OR "i dread" OR "i am horrified" OR "i worry" OR "i feel afraid" OR "i feel scared" OR "i am terrified" OR "i feel terrified" OR "i feel worried" OR "worries me" OR "scares me" OR "frightens me" OR "horrifies me" OR "terrifies me") (libtards OR democrats OR liberals) lang:en -is:retweet First Person Tweet Collection
  28. 28. The Archives in the Pilot
  29. 29. Deduplication without Retweets
  30. 30. Checking Inter-Rater Reliability •We conducted four reliability checks •Datasets were 200, 200, 100, and 200 items •We used between 6 & 12 coders •Fleiss’ kappa = .76, .91, .80, and .85
  31. 31. Checking Validity •We conducted regular validity checks •Thousands of observations were validated •Very few invalid observations overall •Invalid observations not used for training •Better quality training data •The “gold standard” •Better understanding in the 50+ page codebook
  32. 32. • We rank coders all the time. • CoderRank is the notion that for any annotation task, simple to complex, there is a range of human aptitude. • A small number of coders are fantastic. • Surprisingly small at times. • A larger number is awful. • Especially for hard tasks! • Most are average. • ~65-85% valid. CoderRankSM
  33. 33. 5732814
  34. 34. Dr. Stuart W. Shulman Founder & CEO, Texifter, LLC Editor Emeritus, Journal of Information Technology & Politics stu@texifter.com @stuartwshulman Thank-you for having me!

×