This document introduces text analysis and provides an overview of how to perform text analysis. It defines text analysis as using computational tools to quickly search large texts and conduct complex searches. The document explains that text analysis can help test intuitions during research by encouraging reflection on questions asked of texts. Several examples of text analysis tools and projects are provided, such as Voyant Tools, Google Ngram Viewer, and projects analyzing literary texts. A variety of open-source text analysis tools and programming languages are introduced.

1. Introduction to Text Analysis
MLA Annual Convention
Getting Started in the Digital Humanities
January 9, 2014
Lauren F. Klein
Georgia Institute of Technology
lauren.klein@lmc.gatech.edu
@laurenfklein

2. Introduction to Text Analysis
• What is text analysis?

3. Introduction to Text Analysis
• What is text analysis?
• Why should you use it?

4. Introduction to Text Analysis
• What is text analysis?
• Why should you use it?
• How do you use it?
– Examples
– Tools

5. What is Text Analysis?

6. What is Text Analysis?
According to Geoffrey Rockwell:
•
“Text analysis systems can search large texts quickly. They do this by preparing
electronic indexes to the text so that the computer does not have to read through
the entire text. When finding words can be done so quickly that it is "interactive",
it changes how you can work with the text - you can serendipitously explore
without being frustrated by the slowness of the search process.
•
“Text analysis systems can conduct complex searches. Text analysis systems will
often allow you to search for lists of words or for complex patterns of words. For
example you can search for the co-occurrence of two words.
•
“Text analysis systems can present the results in ways that suit the study of
texts. Text analysis systems can display the results in a number of ways; for
example, a Keyword In Context display shows you all the occurrences of the found
word with one line of context.”
http://tada.mcmaster.ca/Main/WhatTA

8. http://www.wordle.net

10. http://www.wordle.net

12. Why Use Text Analysis?

13. Why Use Text Analysis?
Geoff Rockwell, again:
•
•
•
“Text analysis tools aide the interpreter asking questions of electronic texts.”
“Text analysis practices encourage reflection on the questions asked and
formalization of queries.”
“Text analysis is a way of targeting rereading that tests intuitions.”

14. Why Use Text Analysis?
Geoff Rockwell, again:
•
•
•
“Text analysis tools aide the interpreter asking questions of electronic texts.”
“Text analysis practices encourage reflection on the questions asked and
formalization of queries.”
“Text analysis is a way of targeting rereading that tests intuitions.”

15. Why Use Text Analysis?
Geoff Rockwell, again:
•
•
•
“Text analysis tools aide the interpreter asking questions of electronic texts.”
“Text analysis practices encourage reflection on the questions asked and
formalization of queries.”
“Text analysis is a way of targeting rereading that tests intuitions.”
Ted Underwood:
• “Proving a literary thesis with statistical analysis is often like cracking a nut with a
jackhammer. You can do it: but the results are not necessarily better than you
would get by hand.”

16. Why Use Text Analysis?
Geoff Rockwell, again:
•
•
•
“Text analysis tools aide the interpreter asking questions of electronic texts.”
“Text analysis practices encourage reflection on the questions asked and
formalization of queries.”
“Text analysis is a way of targeting rereading that tests intuitions.”
Ted Underwood:
• “Proving a literary thesis with statistical analysis is often like cracking a nut with a
jackhammer. You can do it: but the results are not necessarily better than you
would get by hand.”
What I think (in the spirit of Movable Type):
• Text analysis as “a way to tell a new story.”

17. How to Use Text Analysis?

18. Ben Blatt,
http://www.slate.com/articles/arts/culturebox/2013/11/hunger_games_catching_fire_a_text

19. Sarah Lohman,
http://www.fourpoundsflour.com/the-gallery-data-visualization-of-a-timeline-of-taste/

20. Daniel, http://lkleincourses.lmc.gatech.edu/dh12/2012/02/22/the-role-of-senses-in-a-studyin-scarlet/

21. Ted Underwood and Jordan Sellers, http://journalofdigitalhumanities.org/1-2/theemergence-of-literary-diction-by-ted-underwood-and-jordan-sellers/

22. Rob Nelson, http://dsl.richmond.edu/dispatch/

23. Matt Jockers, http://www.nbcnews.com/technology/data-mining-classics-makes-beautifulscience-954577

24. Matt Jockers, from Macroanalysis (Univ. of Illinois Press, 2013)

25. Lauren Klein, from “The Image of Absence” (American Literature 85.4)

26. Tools for Text Analysis
•
•
•
•
•
•
•
•
Wordle
Google Ngram Viewer
IBM Many Eyes
Voyant
MONK (requires institutional access)
MALLET
Stanford’s Natural Language Processing Toolkit
R

27. Google Ngram Viewer
Google Ngram Viewer
https://books.google.com/ngrams

28. IBM Many Eyes
Many Eyes
http://www-958.ibm.com/software/analytics/manyeyes/

29. Voyant Tools
Voyant Tools
http://voyant-tools.org/

30. MALLET
MALLET
http://mallet.cs.umass.edu/

31. Stanford NLP Toolkit
Stanford NLP Toolkit
http://nlp.stanford.edu/downloads/

32. R Programming Language
R (programming language)
http://www.r-project.org/

33. TAPoR
TAPoR (Text Analysis PoRtal)
http://tapor.ca/

34. More Lists of Tools
• http://toolingup.stanford.edu/?page_id=367
• http://guides.library.upenn.edu/dhtextanalysi
s
• http://dirt.projectbamboo.org/categories/text
-mining

35. Many Eyes Demo
http://lkle.in/1bTr2eT

36. Voyant Tools Demo
http://lkle.in/1e186zN