Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
NASIG 2017
Technical Services and
the Virtual Reference Desk:
Mining Chat Transcripts for
Improved E-Resource
Management
J...
About Georgetown University
 18,000 students
2,500 full- and part-time faculty
 Additional GU Library locations:
– Downt...
About the GU Library
 Collections:
– 190,000 e-journals
– 1,236 “databases”
– 1.5m e-books
 Systems
– ILS: III/Sierra
– ...
Chat at GU Library
 “Ask Us” chat widgets on Library website, some LibGuides, and Summon
 Libraryh3lp chat software unde...
Chat Dataset
 1,898 chats in “Fall 2016”
– Aug 1 – Dec 31 2016
Right: We downloaded text files
from Libraryh3lp, then use...
Chat Dataset
 1,898 chats in “Fall 2016”
– Aug 1 – Dec 31 2016
Below: Excel sheet we used to
categorize observations
Chat Dataset: A Rubric
 Did chat involve an electronic resource (ER)?
– Was patron explicitly looking for an online datab...
ER-Related Chats: Follow-up Questions
If ER-related:
 What resource(s) were involved?
 What was the outcome?
 Was the q...
05:35PM 97126954770937742084319084@web.libraryh3lp.com Hi, I was
wondering if I have access to the Economist Intelligence ...
03:49PM 31727676400954745276307694@web.libraryh3lp.com Hello. If I
pay for guest access (I think it's $300?), will I have ...
Chats: Non-ER vs ER-Related
ER-
related,
551
(29%)
Other,
1347
(71%)
Known-item,
479, 87%
1,898 total chats
(Aug 1 – Dec 3...
Chats: What resource?
 320 distinct resources (databases or journal titles)
Resource Count
Taylor & Francis titles 31
Har...
Chats: Results and Referral status
Success
207
No sub
or Print
only
183
Access
issue
91
No
access
15
Staff
error
16
Unansw...
Chats: Results and Referral status
Not
Referred
482
Referred
42
Should/Maybe
27
Referred
Right: Chats by referral status. ...
Word Frequency Analysis
 ER-related subset
 Patron chat text only
 Customized stopwords
We used Voyant text processing ...
Word Frequency Analysis
access
trying
Term Count
access 500
article 333
library 212
journal 182
help 176
online 165
trying...
Takeaways for ER&S
 Distinctions between a “database” or journal and a collection
of databases / journals is confusing fo...
Takeaways for ER&S
 Many patrons asking about certain frequently named sources (Wall Street Journal,
etc.)
…8116@web.libr...
Takeaways for ER&S
 Many patrons asking about certain frequently named sources (Wall Street Journal,
etc.)
– Make “quick ...
Takeaways for ER&S
 Some resources need clearer “lines of support”
– Refworks: support split between Reference and E-Reso...
Takeaways (Post-Migration to Alma/Primo)
 Consortial e-books: Patrons were confused about e-books held
by other instituti...
Takeaways (Post-Migration to Alma/Primo)
 Consortial e-books: Patrons were confused about e-books held
by other instituti...
Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libra...
Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libra...
Takeaways (Post-Migration to Alma/Primo)
 URLs in 856 that do not lead to full text
– Aka “856 confusion”
…3707@web.libra...
Questions We’re Pondering
 Should our discovery tools mention frequently requested resources,
even if we don’t own them?
...
Ending on a High Note
…7287@web.libraryh3lp.com: Who could I write to thank for this
great search tool One Search [Summon]...
Thank you!
Questions?
John Kimbrough
Electronic Resources
Georgetown University
jwk77@georgetown.edu
Above: Aerial shot of...
Upcoming SlideShare
Loading in …5
×

Technical Services and the Virtual Reference Desk: Mining Chat Transcripts for Improved E-Resource Management

260 views

Published on

Where do patrons encounter problems with our e-resources? With many libraries using online chat to communicate with patrons, chat transcripts provide a valuable source of data to improve the user experience. Although public services staff often review transcripts, there are lessons in the transcripts for technical services librarians as well. At Georgetown University, we analyzed transcripts from Libraryh3lp chat software to identify frequent e-resource problems, improve our processing workflows and collaborate with our public services colleagues.

Speaker: John Kimbrough, Georgetown University Library

Published in: Education
  • Be the first to comment

Technical Services and the Virtual Reference Desk: Mining Chat Transcripts for Improved E-Resource Management

  1. 1. NASIG 2017 Technical Services and the Virtual Reference Desk: Mining Chat Transcripts for Improved E-Resource Management John Kimbrough Georgetown University Library
  2. 2. About Georgetown University  18,000 students 2,500 full- and part-time faculty  Additional GU Library locations: – Downtown D.C. – Doha, Qatar  Member of Washington Research Library Consortium – Shared discovery – Shared e-book purchases  Jesuit university founded in 1789, located in Washington, D.C. Georgetown Library in Doha, Qatar Additional text for Slideshare version in maroon italic
  3. 3. About the GU Library  Collections: – 190,000 e-journals – 1,236 “databases” – 1.5m e-books  Systems – ILS: III/Sierra – Discovery: Summon – E-Journals: Ex Libris 360 (Serials Solutions) – Database A-Z: LibGuides – Moving to Alma/Primo in summer 2018 Students studying in Lauinger Rachel Shaar, The Hoya
  4. 4. Chat at GU Library  “Ask Us” chat widgets on Library website, some LibGuides, and Summon  Libraryh3lp chat software underlies widget – Automatically records transcripts of all chats  Chats answered by public services staff – Reference librarians – Full-time access services staff – Student employees at circulation  ER staff (us) not normally on chat – Should ER staff help staff chat? Above: Examples of chat widgets used at Georgetown.
  5. 5. Chat Dataset  1,898 chats in “Fall 2016” – Aug 1 – Dec 31 2016 Right: We downloaded text files from Libraryh3lp, then used Access to combine chats into a single PDF (snippet shown)
  6. 6. Chat Dataset  1,898 chats in “Fall 2016” – Aug 1 – Dec 31 2016 Below: Excel sheet we used to categorize observations
  7. 7. Chat Dataset: A Rubric  Did chat involve an electronic resource (ER)? – Was patron explicitly looking for an online database? – Was patron looking for a journal article?  presume ER, unless mention print – Was patron looking for a book?  not ER, unless mention e-version …3217@web.libraryh3lp.com: I'm looking for a book by Herbert Goldhamer entitled The Advisor. I can't figure out how to use the on line process …1035727@web.libraryh3lp.com: hello there! i was wondering if Lauinger [had] The San Francisco Chronicle [Final? Edition] available in microform from 1991? • If chat was ER-related, move on to further questions…. The SF Chronicle chat wasn’t ER-related, because patron asked about microfilm. The Goldhamer book also wasn’t ER- related, because the patron didn’t specify an online edition (just that he/she was having difficulties searching)
  8. 8. ER-Related Chats: Follow-up Questions If ER-related:  What resource(s) were involved?  What was the outcome?  Was the question referred (or should have been referred) to ER staff?  Anything we (ER staff) could learn to improve processing and procedures? …48826@web.libraryh3lp.com: Adobe Digital is asking me to Select eBook vendor, then enter your login ID and password. Librarian: OK - one moment. I'm going to see if I can get one of our eresources people on the line to help with this.
  9. 9. 05:35PM 97126954770937742084319084@web.libraryh3lp.com Hi, I was wondering if I have access to the Economist Intelligence Unit database through georgetown? 05:36PM gt-circdesk@libraryh3lp.com One moment, let me check. Are you currently a Georgetown student? 05:36PM 97126954770937742084319084@web.libraryh3lp.com Yes 05:37PM gt-circdesk@libraryh3lp.com It looks like we do have access. If you look at our alphabetical list of databases you will find it by scrolling down: http://guides.library.georgetown.edu/az.php?a=e 05:37PM 97126954770937742084319084@web.libraryh3lp.com perfect thanks 10:53AM 94238908719533669698653479@web.libraryh3lp.com How will I be notified when a book I requested for pick up is ready? 10:53AM gt-circdesk@libraryh3lp.com You will receive an email. 10:54AM 94238908719533669698653479@web.libraryh3lp.com thank you! 10:54AM gt-circdesk@libraryh3lp.com :-) Example chats. Left: an ER-related chat coded as “success.” Below: a non-ER-related chat.
  10. 10. 03:49PM 31727676400954745276307694@web.libraryh3lp.com Hello. If I pay for guest access (I think it's $300?), will I have access to online academic databases such as J-STOR? 03:49PM 31727676400954745276307694@web.libraryh3lp.com (from off- campus, I mean) 03:50PM gt-circdesk@libraryh3lp.com Hi there, unfortunately $300 library associate membership does not grant you off-campus database access. You would have to come to the library and use one of our guest computers to access our databases 03:51PM 31727676400954745276307694@web.libraryh3lp.com Shoot. I'm trying to finish a paper from California and am struggling to get article access anywhere. Thanks. Another example chat (above) from an alum.
  11. 11. Chats: Non-ER vs ER-Related ER- related, 551 (29%) Other, 1347 (71%) Known-item, 479, 87% 1,898 total chats (Aug 1 – Dec 31, 2016) ER-Related chats: Other chat types included: – Alum / public access from off-campus – General usability – Feedback or solicitations
  12. 12. Chats: What resource?  320 distinct resources (databases or journal titles) Resource Count Taylor & Francis titles 31 Harvard Business Review or HBS Cases 24 Wiley titles 20 Summon/360 19 ProQuest (inc. Ebrary, Safari) 17 Wall Street Journal 12 EBSCO 12 SAGE titles 11 Refworks 11 Univ. of Chicago Press 10 Elsevier titles 9 Foreign Affairs 9 New York Times 9 ArcGIS 9 Resource Count Economist or EIU 8 JSTOR 8 Noodletools 6 Washington Post 6 Oxford University Press 5 Ovid 5 Project Muse 5 Bloomberg 4 Financial Times 4 Top 22 Resources Total: 261 (47% of total ER responses) Red borders: news sources. We were surprised at how many news sources were in the chart. Green arrows: Refworks (expected, we use Refworks heavily) and Noodletools (unexpected) Left: Table showing the top 23 most chatted- about resources (with some grouping). Chart is broken into two parts.
  13. 13. Chats: Results and Referral status Success 207 No sub or Print only 183 Access issue 91 No access 15 Staff error 16 Unanswered/other 44 Results Left: chats by result. For ER processing, we consider both “success” and “No sub” (patron referred to ILL) as successful.
  14. 14. Chats: Results and Referral status Not Referred 482 Referred 42 Should/Maybe 27 Referred Right: Chats by referral status. “Referred” = library staff gave out ER contact info, or said they would contact ER during chat. Some chats were not referred, but analysts thought they should have been referred. This chart (and one on preceding page) demonstrated to us that a regular ER staff presence on chat was not required: most chats had a successful result, and were not referred.
  15. 15. Word Frequency Analysis  ER-related subset  Patron chat text only  Customized stopwords We used Voyant text processing software as an additional check on our manual coding. Right: “Cirrus cloud” produced by Voyant.
  16. 16. Word Frequency Analysis access trying Term Count access 500 article 333 library 212 journal 182 help 176 online 165 trying 155 link 137 looking 125 search 116 just 115 need 111 Term Count georgetown 107 articles 100 know 100 available 88 book 87 campus 87 Left and above: Frequent terms in ER corpus. Right: Link chart – words appearing frequently in proximity to “article” or “book.” Green circles = words associated with article. Uncircled = associated with book. Red boxes = associated with both.
  17. 17. Takeaways for ER&S  Distinctions between a “database” or journal and a collection of databases / journals is confusing for patrons and staff – Include links to content collections (e.g. “EBSCO”) in our A-Z list and other discovery tools, even if this is redundant …3376@web.libraryh3lp.com: Hi how do I access EBSCOhost? Librarian: Ebsco is a company. Which EBSCO product are you talking about? …6482@web.libraryh3lp.com: how do i access the taylor and francis online? Librarian: let me look into that
  18. 18. Takeaways for ER&S  Many patrons asking about certain frequently named sources (Wall Street Journal, etc.) …8116@web.libraryh3lp.com: Is there any way I can access Wall Street Journal online? …8821a@web.libraryh3lp.com: I am searching for the following WSJ article: http://www.wsj.com/articles/in-afghanistan- a-cagney-searches-for-her-lacey- 1480696811 …0082@web.libraryh3lp.com: hi, is it possible to read wall street journal using the university's credential without having to sign in/subscribe to the WSJ web?
  19. 19. Takeaways for ER&S  Many patrons asking about certain frequently named sources (Wall Street Journal, etc.) – Make “quick links” to these sources more visible – Document for chat responders exactly what we have (and don’t have) for these sources • No access to Economist Espresso, Audio Edition, etc. • Browse: Economist.com, ABI/Inform • Search: Factiva, ABI/Inform • No access to wsj.com • Browse: Factiva News Pages • Search: Factiva, ABI/Inform
  20. 20. Takeaways for ER&S  Some resources need clearer “lines of support” – Refworks: support split between Reference and E-Resources – ArcGIS: split between E-Resources, Library IT, and University IT …4244@web.libraryh3lp.com: Hi there! I can't get into my RefWorks account, even when I'm logged in to the proxy network.  Several small/miscellaneous things for e-resources staff to fix (e.g., holdings updates) – Some fixed right after chat, some fixed independently as part of normal procedures …3707@web.libraryh3lp.com: Hello. What email should I use to access the Armed Conflict Database? […] …3707@web.libraryh3lp.com: Right It says that the site is not a configured remote resourceRight: “not a configured remote resource” appears on the error page of our proxy server.
  21. 21. Takeaways (Post-Migration to Alma/Primo)  Consortial e-books: Patrons were confused about e-books held by other institutions in Summon – Example book Left: screenshot of an ebook from our Summon instance
  22. 22. Takeaways (Post-Migration to Alma/Primo)  Consortial e-books: Patrons were confused about e-books held by other institutions in Summon – Example book Above: the catalog page (from WRLC) that displays when the “full text online” link from Summon is clicked (see previous slide). There is a limit displayed (in yellow), but also another “full text” link, which is confusing for patrons.
  23. 23. Takeaways (Post-Migration to Alma/Primo)  URLs in 856 that do not lead to full text – Aka “856 confusion” …3707@web.libraryh3lp.com: I have a question about the book - Colonial Effects: The Making of National Identity in Jordan. The record shows there is an ebook but the link goes to another website Above: search result screen from the current Georgetown OPAC, displaying an “Electronic Resource” link.
  24. 24. Takeaways (Post-Migration to Alma/Primo)  URLs in 856 that do not lead to full text – Aka “856 confusion” …3707@web.libraryh3lp.com: I have a question about the book - Colonial Effects: The Making of National Identity in Jordan. The record shows there is an ebook but the link goes to another website Left: detailed result screen from Georgetown OPAC. The “Electronic Resource” link (previous screen) was triggered by the presence of an 856 tag in the MARC record…
  25. 25. Takeaways (Post-Migration to Alma/Primo)  URLs in 856 that do not lead to full text – Aka “856 confusion” …3707@web.libraryh3lp.com: I have a question about the book - Colonial Effects: The Making of National Identity in Jordan. The record shows there is an ebook but the link goes to another website …but the URL provides information about the donor, because the item was purchased on an endowed fund. No full text available.
  26. 26. Questions We’re Pondering  Should our discovery tools mention frequently requested resources, even if we don’t own them?  How can we help public services colleagues handling chats? – Invited to present to circ staff in late June, possibly again in Oct …2332@web.libraryh3lp.com: Hi! I was just wondering if Georgetown had an account with noodle tools Librarian: No, we don't. We only subscribe to RefWorks …2332@web.libraryh3lp.com: Is that similar?
  27. 27. Ending on a High Note …7287@web.libraryh3lp.com: Who could I write to thank for this great search tool One Search [Summon]. I am able to locate my books and papers so easily …7287@web.libraryh3lp.com: I really want to thank someone, because this has been such a breeze and it's so helpful. …2528@web.libraryh3lp.com: And i am using this search and i am loving it. My books show up immediately -- i am simply copying from the syllabus and boom the search show my items …2528@web.libraryh3lp.com: i wanted to let someone know how helpful this has been and I am so excited to have this search option, onesearch, available to use. Thank you for having it. Left and Below: Many chats dealt with problems, but there were a few patrons who wanted to thank the library for providing wonderful tools and services.
  28. 28. Thank you! Questions? John Kimbrough Electronic Resources Georgetown University jwk77@georgetown.edu Above: Aerial shot of Georgetown campus, Georgetown University Slide 1: “Man panning gold on Nome Beach”, Library of Congress. https://www.loc.gov/item/99614756/ Thank you to Mark Winek, Vani Murthy, Shu-Chen Tsung, and GU ER&S staff!

×