1. Sifting Social Data
Word Sense Disambiguation Using Machine Learning
Dr. Stuart Shulman
Founder & CEO, Texifter
“…a wealth of information creates a poverty of attention.”
- Herbert Simon, 1971

2. Pronounced “tech-sifter” the metaphor is of a sifter

3. Text Classification
A 2500 year-old problem
Plato argued it would be frustrating and it still is…

4. Grimmer & Stewart “Text as Data” Political Analysis (2013)
Volume is a problem for scholars
Coders are expensive
Groups struggle to accurately label text at scale
Validation of both humans and machines is “essential”
Some models are easier to validate than others
All models are wrong
Automated models enhance/amplify, but don’t replace humans
There is no one right way to do this
“Validate, validate, validate”
“What should be avoided then, is the blind use of any method without a validation step.”

5. Our free, open-source, web-based text analytics toolkit

6. The original software kernel: tools for measurement

7. A mission to avoid tennis elbow
Items load to the screen and the coder hits the keystroke

8. Keystroke human coding: alone or in groups
Codes
Metadata
Data
Human coding can be distributed to individuals, groups & crowds

9. Computer science & NSF influences: measure everything
How fast?
How reliable?
How accurate?
Stuart Shulman – Texifter

10. Inter-rater reliability is one critical measurement
Stuart Shulman – Texifter

12. Plugged in to APIs & Government

13. Import data directly via APIs or from your desktop
Stuart Shulman – Texifter

14. Full historical Twitter access
Stuart Shulman – Texifter

15. PowerTrack operators for more precise queries
Stuart Shulman – Texifter

16. Store social data with survey responses and other data
Stuart Shulman – Texifter

17. Private, 3rd party & free (rate limited) social data sources
Stuart Shulman – Texifter

18. Unlimited “fire hose” premium data sources
Stuart Shulman – Texifter

19. The Five Pillars of Text Analytics
Search
Filtering
De-duplication and Clustering
Human Coding
Machine-Learning
Stuart Shulman – Texifter

20. Pillar #1: Search
Stuart Shulman – Texifter

21. Pillar #1: Defined multi-term search
Stuart Shulman – Texifter

22. Pillar #2: Filters
Stuart Shulman – Texifter

23. Pillar #2: Filters
Stuart Shulman – Texifter

24. Pillar #3: Deduplication & clustering
Stuart Shulman – Texifter

25. Pillar #3: Deduplication & clustering
Stuart Shulman – Texifter

26. Pillar #4: Human coding (a.k.a. labeling or tagging)
Stuart Shulman – Texifter

27. Pillar #4: Human coding
Stuart Shulman – Texifter

28. Pillar #4: Human coding (adjudication)
Stuart Shulman – Texifter

29. Pillar#5: Machine-learning
Stuart Shulman – Texifter

30. Pillar#5: Machine-learning
Stuart Shulman – Texifter

31. Our ActiveLearning engine and coding tools combine…
what humans do best… with what computers do best
Humans and machines learning together
Keep humans “in-the-loop” for more accurate results and better insights
Stuart Shulman – Texifter

32. Word sense disambiguation (relevance)
Stuart Shulman – Texifter

33. Word sense disambiguation (relevance)
Stuart Shulman – Texifter

34. Word sense disambiguation (relevance)
Stuart Shulman – Texifter

35. Stuart Shulman – Texifter

36. Human coding can be converted into machine classifiers
Accumulated human coding becomes training data via machine-learning

37. Users can drill into interactive reporting displays
Use metadata to examine sub-sets of responses and create reports.

38. Slicing big piles of text into smaller, more focused sets is key
Ultimately all text analytics are filtering techniques

39. Crowdsourcing accelerates the insight generation process through machine-learning
Distributed for synchronous & asynchronous collaboration

40. CoderRank (patent pending) for enhanced machine-learning is our key innovation

41. For more information visit the Texifter table or
discovertext.com
@discovertext
Thank-you for listening!
Stuart Shulman – Texifter