The document discusses priority areas for researching the value and impact of academic libraries. It identifies the key areas as communication, mission alignment, learning analytics, student success, teaching and learning, and collaboration. For each area, it provides exemplar effective practices from literature and interviews with librarians and administrators. It then outlines potential research questions within each area and discusses research design considerations. The document concludes with an overview of a visualization tool being developed to showcase findings.

1. Value
of Academic Libraries
ALA | 25 June 2017
Academic Library Impact:
Improving Practice and Essential Areas to Research
Lynn Silipigni Connaway, Senior Research Scientist & Director of User Research, OCLC
William Harvey, Consulting Engineer, OCLC
Vanessa Kitzie, Assistant Professor, University of South Carolina
Stephanie Mikitish, Librarian, Rutgers University

2. Value
of Academic Libraries
INTRODUCTION

3. Value
of Academic Libraries
The Value of Academic
Libraries
• ACRL Goal-area committee
– Part of Plan for Excellence
• Goal: Academic libraries
demonstrate alignment with &
impact on institutional
outcomes
– Promote impact & value of
libraries to higher ed. community

4. Value
of Academic Libraries
Selected Presentations
• Library Assessment
Conference (Nov 2016)
– Results of literature review
• ALA Midwinter (Jan 2017)
– Results of librarian and provosts
interviews
• ACRL (March 2017)
– Priority areas and preliminary
research questions
– Visualization tool

5. Value
of Academic Libraries
Agenda
• Priority areas
– Effective practices from
literature and interviews
– Research questions
– Exemplar studies for
content and research
design
• Visualization tool
• Final steps

6. Value
of Academic Libraries
Priority Areas
1. Communication
2. Mission strategy & alignment
3. Learning analytics
4. Student success
5. Teaching & learning
6. Collaboration

7. Value
of Academic Libraries
PRIORITY AREAS:
EFFECTIVE PRACTICES

8. Value
of Academic Libraries
Communicate the Library’s Contributions
Host town halls that address the
role of libraries (PP04) and work
to dismantle “pigeonholed”
perceptions
Emphasize the library’s ability to
have “impacts beyond borders”
(PP01) when making funding
requests
Give provosts a “different style of
data” (PP11)

9. Value
of Academic Libraries
Match Library Assessment to Institution’s Mission
Be open to adopting less traditional
roles, such as creating 3 minute
research video competitions
(LM03 and PP03)
Be aware of student demographics,
and accommodate students with
low SES to demonstrate the
importance of the “university as a
gateway to a better life” (PP08)

10. Value
of Academic Libraries
Include Library Data in Institutional Data Collection
Consider collecting holistic student
data before and after graduation
(PP03, PP11)
Refer to data collected by independent,
for-profit institutions (PP14) for
suggestions
Engage senior leadership in dialogue
about the institution’s data
environment (LM13)

11. Value
of Academic Libraries
Quantify the Library’s Impact on Student Success
Measure, collect, analyze and report
data collected from multiple
systems within the academic
institution and statistically analyzed
to predict student success.
Select student success indicators that
consider the diverse experiences of
students.

12. Value
of Academic Libraries
Enhance Teaching and Learning
Collaborate across departments to
leverage library services, spaces,
collections, and expertise to assist
students, perhaps in first-year
experience programs (LM06, PP08)
Keep abreast of higher education
trends, such as fostering the critical
competency skills (PP03, PP10)
needed to be “informed citizens”
(PP06, PP14)

13. Value
of Academic Libraries
Collaborate with Educational Stakeholders
Provide library space and expertise to
support activities such as
interdisciplinary work (PP03, PP08)
or career counseling (PP05)
Partner with institutions outside the
university or college, such as
government and commercial
institutions or community groups
(PP13)

14. Value
of Academic Libraries
PRIORITY AREAS:
RESEARCH QUESTIONS

15. Value
of Academic Libraries
• How are other units effectively
communicating with stakeholders?
• What factors influence librarian
communication between academic library
users and potential users?
• How can library administrators and staff
leverage social media to effectively
increase student engagement?
Research Questions:
Communication

16. Value
of Academic Libraries
• How do libraries fit into the broader array
of institutional resources and programs
(e.g., writing centers, tutoring)?
• How are budget constraints affecting the
support by library administrators and staff
of the institution’s mission and specific
goals related to student learning and
success outcomes?
Research Questions:
Mission strategy &
alignment

17. Value
of Academic Libraries
• How are other stakeholders in
higher education using
analytics to affect the areas of
teaching and learning and
student success, and how can
library administrators and staff
contribute to these efforts?
• Will library administrators and
staff need to begin collecting
different and additional data?
Research Questions:
Learning analytics

18. Value
of Academic Libraries
• How do library resources and programs
(i.e., courses, events, etc.) impact
indicators of student success?
• What are the effects of library instruction
on success outcomes for diverse student
populations, (e.g., military students, non-
US students, English language learners,
non-residential students, online-only
students, etc.)?
Research Questions:
Student Success

19. Value
of Academic Libraries
• What are the most common difficulties
faced by library administrators and
staff in measuring teaching and
learning outcomes?
• Where do students go to discover
information (e.g., answers to
questions, resources for their needs)?
If this is not library websites or online
catalogs, library discovery systems be
integrated into academic users’
workflows?
Research Questions:
Teaching & Learning

20. Value
of Academic Libraries
• How can library administrators and
staff collaborate with staff and faculty
members from other academic
institutions to increase student
learning and success?
• What types of collaboration are the
most effective in facilitating student
learning and success outcomes?
Research Questions:
Collaborate

21. Value
of Academic Libraries
PRIORITY AREAS:
RESEARCH DESIGN

22. Value
of Academic Libraries
Preliminary work
• Context
– Audience
– Prior knowledge,
experience, & bias
– Environment
• Topic/problem
– Past work
– Gaps in research

23. Value
of Academic Libraries
Planning the study
• Design
– Qual/quant elements
– Triangulation
– Clear explanation

24. Value
of Academic Libraries
Data
• Sampling
– Who
– Recruitment
– Gatekeepers
• Collection
– Triangulation
– Rationale
– Relevant & critical

25. Value
of Academic Libraries
Data – ctd.
• Analysis
– Answer RQs
• Modification
– Appropriateness
– Necessary standard

26. Value
of Academic Libraries
Write-up
• Discussion
– Clear, relevant
findings
– Reiterate & frame
– Compare to past
work
• Future work
– Limitations
– Implications

27. Value
of Academic Libraries
Write-up – ctd.
• Reporting
– Non-LIS venues
– Clarity
– Relevance

28. Value
of Academic Libraries
VISUALIZATION COMPONENT

29. Value
of Academic Libraries
Overview
• Quick Recap of
Visualization
Component
• About Usability Testing
• How We Did It
• What We Learned
• New Features

30. Value
of Academic Libraries
Visualization Component
• Literature Search Tool
– “Zappos for scholarly articles”
– Navigate document database
– Build/save/distribute/curate
reference lists
• Charts and Graphs Tool
– “Tableau for scholarly articles”
– Generate graphics to share
with stakeholders

31. Value
of Academic Libraries

32. Value
of Academic Libraries

33. Value
of Academic Libraries

34. Value
of Academic Libraries

35. Value
of Academic Libraries

36. Value
of Academic Libraries
What Is Usability Testing?
“Ethnography is the social equivalent of usability
testing. Where usability is about how people directly interact
with a technology in the more traditional sense, ethnography
is about how people interact with each other. As UX
designers, we’re primarily concerned with how we can use
such research to solve a problem through the introduction or
revision of technology.”
-- Nathanael Boehm

37. Value
of Academic Libraries
Usability Testing Reveals Expectations
• A product owner has one set of
expectations
• A software dev team has another set
• N users experiencing the software for the
first time have N sets
• M experienced users have another M sets
• The software itself has another set

38. Value
of Academic Libraries
Usability Testing Reveals Expectations
• A product owner has one set of
expectations
• A software dev team has another set
• N users experiencing the software for the
first time have N sets
• M experienced users have another M sets
• The software itself has another set

39. Value
of Academic Libraries
User Expectations
• Impedance mismatch
– What {users, X} want
– What {users, X} think they
need
– What users really need
– What can be delivered
• Minimize frustration (“It just
works!”) = maximize usability

40. Value
of Academic Libraries
Usability Testing Is Amazing
• The OCLC Usability Lab is a
fantastic resource
• Ethnography is valuable
• Usability Testing is valuable

41. Value
of Academic Libraries
How Many Users to Study?

42. Value
of Academic Libraries
Usability Study Participants
Gender Institution Participant Type Experience with AiA website
F Kent State
Library and Information
Science (student)
None
F
Adams County
Public Library
System
Library and Information
Science Researcher
None
F
California State
University
Dominguez Hills
Practicing Academic Librarian None
F
University of
Dallas
Academic Library
Administrator
Has used a few times
F
University of
Tennessee at
Chattanooga
Practicing Academic Librarian Never
M
Wayne State
University
Library and Information
Science Researcher
Never

43. Value
of Academic Libraries
Usability Tasks
1. Literature search task
only requiring facets
2. Literature search task
only requiring keywords
3. Literature search
combining (1) and (3).
4. Generate a graph
involving aggregation.

44. Value
of Academic Libraries
USABILITY STUDY FINDINGS

45. Value
of Academic Libraries
Literature Search Tasks
• Locating facets was a challenge
– Allow users to search through facets
• Users unsure of what is in the database
– Provide search-as-you-type to close the
loop
• Users want seamless Boolean/keyword
search
– Augment interface so both can coexist

46. Value
of Academic Libraries

47. Value
of Academic Libraries

48. Value
of Academic Libraries

49. Value
of Academic Libraries

50. Value
of Academic Libraries
Charts and Graphs Tool
• Naming things and
terminology is very
important

51. Value
of Academic Libraries
Charts and Graphs Tool
• All users requested some help
materials/videos
– We are working on them
• Some users just are not familiar with
creating charts
– Perhaps training materials + practice
will help
• Familiarity is important
– Give users an escape hatch to Excel

52. Value
of Academic Libraries
ALA Login

53. Value
of Academic Libraries
Novel Features Added
• Sandboxed user uploads
– Log in with ALA account
– Upload new articles (visible only to user)
– Annotate articles, add new annotations
– Save data
• Broad redesign/reflow of Charts and Graphs
tool

54. Value
of Academic Libraries
Conclusion
• Usability Testing optimizes for
the right benchmark: Usability
• Software in the life of the user

55. Value
of Academic Libraries
FINAL REPORT & RELEASE OF
VISUALIZATION COMPONENT:
SEPTEMBER 2017

56. Value
of Academic Libraries
Thank you to the following people for their contributions to this project:
Erin M. Hood, OCLC; Brittany Brannon, OCLC; Marie L. Radford, Rutgers
University; ACRL Board; ACRL VAL Committee; Advisory Group Members

57. Value
of Academic Libraries
References
Association of College and Research Libraries. Forthcoming. Academic Library Impact: Improving
Practice and Essential Areas to Research. Researched by Lynn Silipigni Connaway, William Harvey,
Vanessa Kitzie, and Stephanie Mikitish. Chicago: Association of College and Research Libraries.
Association of College and Research Libraries. 2010. Value of Academic Libraries: A Comprehensive
Research Review and Report. Researched by Megan Oakleaf. Chicago: Association of College and
Research Libraries. http://www.ala.org/acrl/sites/ala.org.acrl/files/content/issues/value/val_report.pdf.
Bloom, Nathanael. 2010. “Ethnography in UX.” UX Matters, June 21.
http://www.uxmatters.com/mt/archives/2010/06/ethnography-in-ux.php.
Brown-Sica, Margaret. 2013. “Using Academic Courses to Generate Data for Use in Evidence Based
Library Planning.” Journal of Academic Librarianship 39, no. 3: 275-87. doi:10.1016/j.acalib.2013.01.001.
Connaway, Lynn Silipigni, and Marie L. Radford. 2017. Research Methods in Library and Information
Science, 6th ed. Santa Barbara, CA: Libraries Unlimited.

58. Value
of Academic Libraries
References
Fister, Barbara. 2010. “Critical Assets: Academic Libraries, A View from the Administration Building.”
Library Journal 135, no. 8: 24-27.
Hess, Amanda Nichols. 2015. “Equipping Academic Librarians to Integrate the Framework into
Instructional Practices: A Theoretical Application.” Journal of Academic Librarianship 41, no. 6: 771-76.
doi:10.1016/j.acalib.2015.08.017.
Jantti, M., and Heath, J. 2016. “What Role for Libraries in Learning Analytics?” Performance
Measurement and Metrics 17, no. 2: 203-10.
Lombard, E. 2012. “The Role of the Academic Library in College Choice.” Journal of Academic
Librarianship 38, no. 4: 237-41.
Wikipedia, s.v. “Usability Testing.” Last modified June 13, 2017.
https://en.wikipedia.org/wiki/Usability_testing.

59. Value
of Academic Libraries
Image Attributions
Slide 3: Image: http://bit.ly/2tF1Mtx (https://www.flickr.com/photos/stiwwe/6238265844/) by Steven Wolf / CC BY-NC 2.0
Slide 4: Image: http://bit.ly/2sPfwFv (https://www.flickr.com/photos/gsfc/3468651131/) by NASA Goddard Space Flight
Center / CC BY 2.0
Slide 5: Image: http://bit.ly/2sivd6R (https://www.flickr.com/photos/scottfeldstein/437129442/) by scott Feldstein / CC BY
2.0
Slide 6: Image: https://www.flickr.com/photos/parksdh/11340519505/ by Daniel Parks / CC BY-NC 2.0
Slide 8: Image: http://bit.ly/2rfLdrx (https://www.flickr.com/photos/corono/2767343921/) by Gareth Davies / CC BY 2.0
Slide 9: Image: http://bit.ly/2srChPm (https://www.flickr.com/photos/collinanderson/2349441498/) by Collin Anderson / CC
BY 2.0
Slide 10: Image: http://bit.ly/2mHdL6q (https://www.flickr.com/photos/pong/2404940312/) by Rob Pongsajapan / CC BY 2.0
Slide 11: Image: http://bit.ly/2scx4ul (https://www.flickr.com/photos/elvissa/505162199/) by Tricia / CC BY 2.0
Slide 12: Image: http://bit.ly/2m9i39Y (https://www.flickr.com/photos/stnorbertcollege/11839410166/) by stnorbert / CC BY-
NC-ND 2.0
Slide 13: Image: http://bit.ly/2rPulGQ (https://www.flickr.com/photos/hoffnungsschimmer/17264753128/) by Lena / CC BY
2.0
Slide 15: Image: http://bit.ly/2s0oo8r (https://www.flickr.com/photos/nseika/8880813114/) by Seika / CC BY 2.0

60. Value
of Academic Libraries
Image Attributions
Slide 16: Image: http://bit.ly/2msQ06a (https://www.flickr.com/photos/governordayton/5572098001/) by Governor Mark
Dayton / CC BY 2.0
Slide 20: Image: http://bit.ly/2scDnhu (https://www.flickr.com/photos/jblyberg/2185724080/) by John Blyberg / CC BY 2.0
Slide 22: Image: http://bit.ly/2rwaKbU (https://www.flickr.com/photos/jeanbaptisteparis/6301697857/) by jeanbaptisteparis /
CC BY-SA 2.0
Slide 23: Image: http://bit.ly/2tFkRMn (https://www.flickr.com/photos/jeanbaptisteparis/6301701751/) by jeanbaptisteparis /
CC BY-SA 2.0
Slide 24: Image: http://bit.ly/2rN05ZK (https://www.flickr.com/photos/beachmobjellies/33241097510/) by beachmobjellies /
CC BY-SA 2.0
Slide 25: Image: http://bit.ly/2sKOzmL (https://www.flickr.com/photos/question_everything/5104987433/) by Let Ideas
Compete / CC BY-NC-ND 2.0
Slide 26: Image: http://bit.ly/2tEVHxh (https://www.flickr.com/photos/kome8/3599392372/) by 8 Kome / CC BY-NC-ND 2.0
Slide 27: Image: http://bit.ly/2tFnNbL (https://www.flickr.com/photos/sakura-harusame/3828276414/) by
PhotographyBySakura / CC BY-NC-ND 2.0
Slide 29: Image: http://bit.ly/2sHxkSr (https://www.flickr.com/photos/marcinignac/7211176930/) by Marcin Ignac / CC BY-
NC-ND 2.0
Slide 30: Image: http://bit.ly/2rILa7G (https://www.flickr.com/photos/hello-sebastian/8207490946/) by Sebastian Sikora /
CC BY 2.0

61. Value
of Academic Libraries
Image Attributions
Slide 36: Image: http://bit.ly/2rNJt4a (https://www.flickr.com/photos/l-i-n-k/3654390818/) by Thomas Link / CC BY-SA 2.0
Slide 38: Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Slide 39: Image: http://bit.ly/2tFB04a (https://www.flickr.com/photos/taspicsvns/15262863623/) by Vern / CC BY-NC-ND 2.0
Slide 40: Image: http://bit.ly/2sHVr3v (https://www.flickr.com/photos/47259373@N02/5170090189/) by Mark Strobl / CC BY
2.0
Slide 41: Image: http://bit.ly/2rNqgQ3 (https://www.flickr.com/photos/75905404@N00/2220590528/) by OZinOH / CC BY-
NC 2.0
Slide 44: Image: http://bit.ly/2ssMcSr (https://www.flickr.com/photos/romanboed/20932834149/) by Roman Boed / CC BY
2.0
Slide 52: Image: http://bit.ly/2sHK7EC (https://www.flickr.com/photos/themeowverlord/13941607659/) by Amy Aletheia
Cahill / CC BY-SA 2.0
Slide 54: Image: http://bit.ly/2sPkLoK (https://www.flickr.com/photos/pedrosz/33385238882/) by Pedro Szekely / CC BY-SA
2.0
Slide 55: Image: http://bit.ly/2rNiAgJ (https://www.flickr.com/photos/monkeypainter/8322729745/) by David Ramos / CC
BY-NC-ND 2.0
Slide 57: Image: http://bit.ly/2rNjU3j (https://www.flickr.com/photos/jenosaur/4051305996/) by jen Collins / CC BY-NC-ND
2.0

62. Thank you!
• Lynn Silipigni Connaway
connawal@oclc.org
@LynnConnaway
• William Harvey
harveyw@oclc.org
• Vanessa Kitzie
kitziev@oclc.org
@vkitzie
• Stephanie Mikitish
mikitish@scarletmail.rutgers.edu