Where do patrons encounter problems with our e-resources? With many libraries using online chat to communicate with patrons, chat transcripts provide a valuable source of data to improve the user experience. Although public services staff often review transcripts, there are lessons in the transcripts for technical services librarians as well. At Georgetown University, we analyzed transcripts from Libraryh3lp chat software to identify frequent e-resource problems, improve our processing workflows and collaborate with our public services colleagues. Speaker: John Kimbrough, Georgetown University Library

1. NASIG 2017
Technical Services and
the Virtual Reference Desk:
Mining Chat Transcripts for
Improved E-Resource
Management
John Kimbrough
Georgetown University Library

2. About Georgetown University
 18,000 students
2,500 full- and part-time faculty
 Additional GU Library locations:
– Downtown D.C.
– Doha, Qatar
 Member of Washington Research Library Consortium
– Shared discovery
– Shared e-book purchases
 Jesuit university founded in 1789, located in Washington, D.C.
Georgetown Library in Doha, Qatar
Additional text for Slideshare version in
maroon italic

3. About the GU Library
 Collections:
– 190,000 e-journals
– 1,236 “databases”
– 1.5m e-books
 Systems
– ILS: III/Sierra
– Discovery: Summon
– E-Journals: Ex Libris 360 (Serials Solutions)
– Database A-Z: LibGuides
– Moving to Alma/Primo in summer 2018
Students studying in Lauinger
Rachel Shaar, The Hoya

4. Chat at GU Library
 “Ask Us” chat widgets on Library website, some LibGuides, and Summon
 Libraryh3lp chat software underlies widget
– Automatically records transcripts of all chats
 Chats answered by public services staff
– Reference librarians
– Full-time access services staff
– Student employees at circulation
 ER staff (us) not normally on chat
– Should ER staff help staff chat?
Above: Examples of chat widgets used at Georgetown.

5. Chat Dataset
 1,898 chats in “Fall 2016”
– Aug 1 – Dec 31 2016
Right: We downloaded text files
from Libraryh3lp, then used Access
to combine chats into a single PDF
(snippet shown)

6. Chat Dataset
 1,898 chats in “Fall 2016”
– Aug 1 – Dec 31 2016
Below: Excel sheet we used to
categorize observations

7. Chat Dataset: A Rubric
 Did chat involve an electronic resource (ER)?
– Was patron explicitly looking for an online database?
– Was patron looking for a journal article?
 presume ER, unless mention print
– Was patron looking for a book?
 not ER, unless mention e-version
…3217@web.libraryh3lp.com:
I'm looking for a book by Herbert
Goldhamer entitled The Advisor.
I can't figure out how to use the
on line process
…1035727@web.libraryh3lp.com:
hello there! i was wondering if
Lauinger [had] The San Francisco
Chronicle [Final? Edition] available
in microform from 1991?
• If chat was ER-related, move on to further questions….
The SF Chronicle chat wasn’t ER-related,
because patron asked about microfilm.
The Goldhamer book also wasn’t ER-
related, because the patron didn’t
specify an online edition (just that
he/she was having difficulties searching)

8. ER-Related Chats: Follow-up Questions
If ER-related:
 What resource(s) were involved?
 What was the outcome?
 Was the question referred (or should have been
referred) to ER staff?
 Anything we (ER staff) could learn to improve
processing and procedures?
…48826@web.libraryh3lp.com: Adobe Digital is
asking me to Select eBook vendor, then enter
your login ID and password.
Librarian: OK - one moment. I'm going to see if I
can get one of our eresources people on the line
to help with this.

9. 05:35PM 97126954770937742084319084@web.libraryh3lp.com Hi, I was
wondering if I have access to the Economist Intelligence Unit database
through georgetown?
05:36PM gt-circdesk@libraryh3lp.com One moment, let me check. Are
you currently a Georgetown student?
05:36PM 97126954770937742084319084@web.libraryh3lp.com Yes
05:37PM gt-circdesk@libraryh3lp.com It looks like we do have access. If
you look at our alphabetical list of databases you will find it by scrolling
down: http://guides.library.georgetown.edu/az.php?a=e
05:37PM 97126954770937742084319084@web.libraryh3lp.com perfect
thanks
10:53AM 94238908719533669698653479@web.libraryh3lp.com
How will I be notified when a book I requested for pick up is
ready?
10:53AM gt-circdesk@libraryh3lp.com You will receive an email.
10:54AM 94238908719533669698653479@web.libraryh3lp.com
thank you!
10:54AM gt-circdesk@libraryh3lp.com :-)
Example chats.
Left: an ER-related chat coded as
“success.”
Below: a non-ER-related chat.

10. 03:49PM 31727676400954745276307694@web.libraryh3lp.com Hello. If I
pay for guest access (I think it's $300?), will I have access to online academic
databases such as J-STOR?
03:49PM 31727676400954745276307694@web.libraryh3lp.com (from off-
campus, I mean)
03:50PM gt-circdesk@libraryh3lp.com Hi there, unfortunately $300 library
associate membership does not grant you off-campus database access. You
would have to come to the library and use one of our guest computers to
access our databases
03:51PM 31727676400954745276307694@web.libraryh3lp.com Shoot. I'm
trying to finish a paper from California and am struggling to get article access
anywhere. Thanks.
Another example chat (above) from an alum.

11. Chats: Non-ER vs ER-Related
ER-
related,
551
(29%)
Other,
1347
(71%)
Known-item,
479, 87%
1,898 total chats
(Aug 1 – Dec 31, 2016)
ER-Related chats:
Other chat types included:
– Alum / public access
from off-campus
– General usability
– Feedback or solicitations

12. Chats: What resource?
 320 distinct resources (databases or journal titles)
Resource Count
Taylor & Francis titles 31
Harvard Business Review or
HBS Cases 24
Wiley titles 20
Summon/360 19
ProQuest (inc. Ebrary, Safari) 17
Wall Street Journal 12
EBSCO 12
SAGE titles 11
Refworks 11
Univ. of Chicago Press 10
Elsevier titles 9
Foreign Affairs 9
New York Times 9
ArcGIS 9
Resource Count
Economist or EIU 8
JSTOR 8
Noodletools 6
Washington Post 6
Oxford University Press 5
Ovid 5
Project Muse 5
Bloomberg 4
Financial Times 4
Top 22 Resources Total: 261
(47% of total ER responses)
Red borders: news sources. We were surprised at how many
news sources were in the chart.
Green arrows: Refworks (expected, we use Refworks heavily) and
Noodletools (unexpected)
Left: Table showing the
top 23 most chatted-
about resources (with
some grouping). Chart
is broken into two parts.

13. Chats: Results and Referral status
Success
207
No sub
or Print
only
183
Access
issue
91
No
access
15
Staff
error
16
Unanswered/other
44
Results
Left: chats by result. For ER processing, we consider both
“success” and “No sub” (patron referred to ILL) as successful.

14. Chats: Results and Referral status
Not
Referred
482
Referred
42
Should/Maybe
27
Referred
Right: Chats by referral status. “Referred” = library
staff gave out ER contact info, or said they would
contact ER during chat. Some chats were not
referred, but analysts thought they should have been
referred.
This chart (and one on preceding page) demonstrated
to us that a regular ER staff presence on chat was not
required: most chats had a successful result, and
were not referred.

15. Word Frequency Analysis
 ER-related subset
 Patron chat text only
 Customized stopwords
We used Voyant text processing software as
an additional check on our manual coding.
Right: “Cirrus cloud” produced by Voyant.

16. Word Frequency Analysis
access
trying
Term Count
access 500
article 333
library 212
journal 182
help 176
online 165
trying 155
link 137
looking 125
search 116
just 115
need 111
Term Count
georgetown 107
articles 100
know 100
available 88
book 87
campus 87
Left and above: Frequent
terms in ER corpus.
Right: Link chart – words appearing frequently in proximity to “article” or “book.”
Green circles = words associated with article.
Uncircled = associated with book. Red boxes = associated with both.

17. Takeaways for ER&S
 Distinctions between a “database” or journal and a collection
of databases / journals is confusing for patrons and staff
– Include links to content collections (e.g. “EBSCO”) in our A-Z list and
other discovery tools, even if this is redundant
…3376@web.libraryh3lp.com: Hi
how do I access EBSCOhost?
Librarian: Ebsco is a company.
Which EBSCO product are you
talking about?
…6482@web.libraryh3lp.com: how do i
access the taylor and francis online?
Librarian: let me look into that

18. Takeaways for ER&S
 Many patrons asking about certain frequently named sources (Wall Street Journal,
etc.)
…8116@web.libraryh3lp.com: Is there any
way I can access Wall Street Journal online?
…8821a@web.libraryh3lp.com: I am
searching for the following WSJ article:
http://www.wsj.com/articles/in-afghanistan-
a-cagney-searches-for-her-lacey-
1480696811
…0082@web.libraryh3lp.com: hi, is
it possible to read wall street journal
using the university's credential
without having to sign in/subscribe to
the WSJ web?

19. Takeaways for ER&S
 Many patrons asking about certain frequently named sources (Wall Street Journal,
etc.)
– Make “quick links” to these sources more visible
– Document for chat responders exactly what we have (and don’t have) for these sources
• No access to Economist Espresso, Audio Edition, etc.
• Browse: Economist.com, ABI/Inform
• Search: Factiva, ABI/Inform
• No access to wsj.com
• Browse: Factiva News Pages
• Search: Factiva, ABI/Inform

20. Takeaways for ER&S
 Some resources need clearer “lines of support”
– Refworks: support split between Reference and E-Resources
– ArcGIS: split between E-Resources, Library IT, and University IT
…4244@web.libraryh3lp.com: Hi there! I can't
get into my RefWorks account, even
when I'm logged in to the proxy network.
 Several small/miscellaneous
things for e-resources staff to fix
(e.g., holdings updates)
– Some fixed right after chat, some
fixed independently as part of
normal procedures
…3707@web.libraryh3lp.com: Hello. What
email should I use to access the Armed
Conflict Database?
[…]
…3707@web.libraryh3lp.com: Right It says
that the site is not a configured remote
resourceRight: “not a configured remote resource”
appears on the error page of our proxy server.

21. Takeaways (Post-Migration to Alma/Primo)
 Consortial e-books: Patrons were confused about e-books held
by other institutions in Summon
– Example book Left: screenshot of an ebook
from our Summon instance

22. Takeaways (Post-Migration to Alma/Primo)
 Consortial e-books: Patrons were confused about e-books held
by other institutions in Summon
– Example book
Above: the catalog page (from WRLC) that displays when the “full text online” link
from Summon is clicked (see previous slide). There is a limit displayed (in yellow),
but also another “full text” link, which is confusing for patrons.

23. Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libraryh3lp.com: I have a question
about the book - Colonial Effects: The Making of
National Identity in Jordan. The record shows
there is an ebook but the link goes to another
website
Above: search result screen from the current Georgetown OPAC,
displaying an “Electronic Resource” link.

24. Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libraryh3lp.com: I have a question
about the book - Colonial Effects: The Making of
National Identity in Jordan. The record shows
there is an ebook but the link goes to another
website
Left: detailed result screen from
Georgetown OPAC.
The “Electronic Resource” link
(previous screen) was triggered by
the presence of an 856 tag in the
MARC record…

25. Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libraryh3lp.com: I have a question
about the book - Colonial Effects: The Making of
National Identity in Jordan. The record shows
there is an ebook but the link goes to another
website
…but the URL provides information
about the donor, because the item
was purchased on an endowed
fund. No full text available.

26. Questions We’re Pondering
 Should our discovery tools mention frequently requested resources,
even if we don’t own them?
 How can we help public services colleagues handling chats?
– Invited to present to circ staff in late June, possibly again in Oct
…2332@web.libraryh3lp.com: Hi! I was just
wondering if Georgetown had an account with
noodle tools
Librarian: No, we don't. We only subscribe to
RefWorks
…2332@web.libraryh3lp.com: Is that similar?

27. Ending on a High Note
…7287@web.libraryh3lp.com: Who could I write to thank for this
great search tool One Search [Summon]. I am able to locate my
books and papers so easily
…7287@web.libraryh3lp.com: I really want to thank someone,
because this has been such a breeze and it's so helpful.
…2528@web.libraryh3lp.com: And i am using this search and i
am loving it. My books show up immediately -- i am simply
copying from the syllabus and boom the search show my items
…2528@web.libraryh3lp.com: i wanted to let someone know
how helpful this has been and I am so excited to have this
search option, onesearch, available to use. Thank you for
having it.
Left and Below: Many chats dealt
with problems, but there were a few
patrons who wanted to thank the
library for providing wonderful tools
and services.

28. Thank you!
Questions?
John Kimbrough
Electronic Resources
Georgetown University
jwk77@georgetown.edu
Above: Aerial shot of Georgetown campus, Georgetown University
Slide 1: “Man panning gold on Nome Beach”, Library of Congress.
https://www.loc.gov/item/99614756/
Thank you to Mark Winek, Vani Murthy, Shu-Chen Tsung, and GU ER&S staff!