This presentation was provided by Jessica Smith of JSTOR Labs during the NISO Hot Topic Virtual Conference "The User Experience: Just Fix It." The event was held on January 26, 2022.

1. A Case Study in User
Needs for Text Analysis
Jessica Smith
Manager, Discovery and Research at JSTOR Labs
January 26, 2022

2. We are a not-for-profit with a mission to improve
access to knowledge and education for people
around the world. We believe education is key to
the well-being of individuals and society, and we
work to make it more effective and affordable.
2

3. Text analysis/mining at ITHAKA
3
JSTOR + Portico =
● ~40,000 journal runs
● ~300,000 books
Started in 2017, ramped up in 2019

4. Background Research
4
● Contextual inquiry-esque interviews with librarians, publishers, and
researchers
● Customer service conversations about JSTOR DfR (Data for
Research)
● Blogs from experts like Ted Underwood and Andrew Goldstone

5. 5

6. Some Findings at This Stage
6
A range of abilities who work completely differently
● Experts who mine text with relative ease
● Learners who can mine text with, usually, 1-on-1 support
● Absolutely beginners who don’t know where to start
. . . and a tension between the groups
The Ivory Tower vs. the Silver Platter

7. Workflow - Sourcing
7
● “Really frustrating to get text ”
● “Data access is the most limiting thing ”
● “I don’t have time for JSTOR yet ”
● This step can take “more than a year ”
● Often not free - might need grant funding

8. Workflow - Structuring
8
● I’ve got an “unknown problem” with an “unknown solution ”
● “You’ve got to expect that manipulations will be required . . . always
bespoke”
● “It’s complicated”
● “There’s no single path”
● “We need interoperability and simplicity”

9. Workflow - Mining and Interpreting
9
In all the steps, it’s harder if you don’t speak this language and TDM so far
is Anglocentric
To publish, they may need preservation of original data and steps to
recreate results
Scholars don’t feel comfortable publishing without understanding, and
being able to defend, the algorithms used

10. Meanwhile, beginners . . .
10
● “Time and anxiety keep me from learning it ”
● “Faculty are too busy to do something new ”
● “DH is a club and it isn’t inclusive ”
● “The real loss isn’t the answers you don’t get, but the questions you
don’t even know to ask ”
● “You can only ask research questions you can imagine the answer to”
There are nice clickable GUI tools like Voyant for analysis, but only meets
a few needs

11. 11

12. And teachers say . . .
12
● “For the disciplines I work in, people are very unfamiliar with numbers
and can’t even look at a spreadsheet ”
● “Most people give up before they start ”
● “It wasn’t a trainwreck. It was worse than that ”
● The naive view is to think this is going to be “like Tableau ”
When teaching someone, ask
1. Do they have the dataset in mind?
2. What’s their tech skill? (e.g. can they scrape)
3. What format do they want the data in?
4. What - if any - are their methods, tools, and research strategy?

13. So, what did we learn?

14. We Learned
14
Sourcing and structuring are the hardest steps
Learning text analysis is hard and Python and R are key
Can humanists learn? Do they want to?
Hard to teach a text mining class at scale
Beginners don’t know what’s possible

15. The term digital humanities has been popular precisely because it
promises that all those projects can still be contained in the
humanities. The implicit pitch is something like this: ‘You won’t
need a whole statistics course. Come to our two-hour workshop on
topic models instead. You can always find a statistician to
collaborate with.’
I understand why digital humanists said that kind of thing eight
years ago. We didn’t want to frighten people away. If you write
‘Learn Another Discipline’ on your welcome mat, you may not get
many visitors. But a deceptively gentle welcome mat, followed
by a trapdoor, is not really more welcoming.
Ted Underwood, January 2018

16. Then, what did we build?

17. 17
Constellate

18. 18

19. Constellate
19

20. Jessica Smith
Manager, Discovery and Research
+1 734 780 2499
jessica.smith@ithaka.org
Thank you
301 E. Liberty
Suite 250
Ann Arbor, MI 48104 labs.jstor.org