Presented at the Virginia Library Association Annual Conference, September 27, 2018

1. Analytics Revolution! Using a
Predictive Model to Measure
the Libraries’ Impact on
Student Success
Joleen McInnis, Health & Life Science Librarian
Lucinda Rush, Instruction Librarian
Leo S. Lo, Associate University Librarian for
Research & Learning
Virginia Library Association Conference
Williamsburg, VA September 27, 2018

2. 2
http://www.creative-commons-images.com/highway-signs/b/big-data.html

3. 3
Analytics and Higher Education

4. 4
“… until libraries know that student #5 with major A
has downloaded B number of articles from
database C, checked out D number of books,
participated in E workshops and online tutorials,
and completed courses F, G, and H, libraries
cannot correlate any of those student information
behaviors with attainment of other outcomes. Until
librarians do that, they will be blocked in many of
their efforts to demonstrate value.”
Megan Oakleaf (2010)
The Value of Academic Libraries
Analytics and Libraries

5. Concerns and Risks
5

6. 6
Cybersecurity
Image Credit: https://pixabay.com/en/hacker-hacking-cyber-security-hack-1944688/

7. 7
Misuse of data
Image Credit: https://www.dailybulletin.com/2018/03/22/facebook-doesnt-want-you-to-worry-about-cambridge-analytica-political-cartoons/

8. 8
Predictive analytics -> self-fulfilling prophecy?
Image credit: https://www.slashgear.com/minority-report-reality-london-police-test-precrime-software-30353166/

9. What is Available to Us
9

10. Student Success Collaborative (SSC): Our View
10

11. SSC: Student View
11
COMING SOON!
Schedule a Library Research Consultation

12. 12
Our Study
 Does participation in library instruction or an
individual research consultation impact:
Course Grade?
Overall GPA?
Retention?
Graduate Rate?
Likelihood of using library services again?

13. 13
1. Getting Started
• Learning how
to use EAB
• IRB
• Seeking help
from partners:
Advising &
Tutoring, ITS,
IRB,
Assessment
2. Participant
Consent
• Faculty
permission
• Info Lit & Data
Lit materials
• Student
consent
3. Add Research
Consultations
• Learning how
to use EAB
(again)
• Training all
liaison
librarians

14. 14
4. Analyze
• Analysis will begin when
we have a full
academic year
• Initial findings may
provide valuable data
that could be used to
impact classroom
teaching
• Full analysis to begin in 5
years.
5. Evaluate
• Is there a correlation
between library
instruction and student
success indicators?
• What have we learned
that an be applied to
our programming and
teaching?
6. Repeat
• What did we learn?
• Are the Results
Consistent?
• How can we apply
what we learn to
improve student
learning outcomes and
positively influence
student achievement?

15. 15
The Institutional Review Board (IRB)

16. 16
IRB Application
•Initially applied as
“exempt”
•Study is actually classified
non-exempt because of
how we will use data and
because of access to
FERPA information.
•Lengthy application
requiring methodologies,
rationale, informed
consent documentation,
etc.
IRB Review
•Feedback from evaluators
•Revisions based on
feedback
•Resubmission
•IRB liaison very helpful in
guiding us through the
rigorous process,
explaining requirements,
and giving a broader
perspective about rules
related to consenting
participants that were not
clear.
Amendments
•Informed Consent Process
refined
•Creation of Informed
consent video to
standardize presentation
of concepts and the study
•Personnel: Needed to
increase librarians able to
consent students
•Update CITI training

17. 17
 Pass out forms
 Show brief video
 Collect all forms
Informed Consent & Privacy

18. Informed Consent Video
18

19. 19
198
Participants, Spring 2018
105
Opted OUT
Opted IN
N=303

20. 20
133
170
64
41
0
20
40
60
80
100
120
140
160
180
English Composition Upper Level Writing Intensive
Participants By Class
Total Students Opt-Ins

21. 21
Future Actions
Add in Research
Consults
Include ALL
teaching
librarians
Identify funds
Data
Management &
Analysis Workflow
Incorporate new
SSC platform &
consider more
uses
Expand Data
Literacy
programming
AND…
• Can we improve the
informed consent
process?
• Are there other
possibilities for campus
partnerships?

22. 22
 Association of College and Research Libraries. Value of Academic Libraries: A Comprehensive Research
Review and Report. Researched by Megan Oakleaf. Chicago: Association of College and Research
Libraries, 2010.
 Butler, Kathy and Jason Byrd. “Research Consultation Assessment: Perceptions of Students and Librarians.” The
Journal of Academic Librarianship, vol. 42, no. 1 , January 2016, pp. 83-86. ScienceDirect,
http://dx.doi.org/10.1016/j.acalib.2015.10.011. Accessed 29 June 2017.
 Haddow, Gaby. “Academic Library Use and Student Retention: A Quantitative Analysis.” Library & Information
Research, vol. 35, no. 2, April 2013, pp.127-136. http://dx.doi.org/10.1016/j.lisr.2012.12.002. Accessed 7
July 2017.
 Matthews, Joseph R. “Assessing Library Contributions to University Outcomes: The Need for Individual Student
Level Data.” Library Management, vol. 33, no. 6/7, Spring 2012, pp. 389-402. EmeraldInsight,
https://doi.org/10.1108/0143512121166203. Accessed 31 October 2017.
 Mezick, Elizabeth M. “Relationship of Library Assessment to Student Retention.” The Journal of Academic
Librarianship, vol. 41, no. 1, January 2015, pp. 31-36. ScienceDirect,
http://doi.org/10.1016.j.acalib.2014.10.011. Accessed 15 January 2018.
 Soria, Krista M., et al. “Library Use and Undergraduate Student Outcomes: New Evidence for Students’
Retention and Academic Success.” portal: Libraries and the Academy, vol. 13, no. 2, April 2013, pp.147-
164. Project Muse, https://doi.org/10.10353/pla.2013.0010. Accessed 31 October 2017.
 Thorpe, Angie, et al. “The Impact of the Academic Library on Student Success: Connecting the Dots. portal:
Libraries and the Academy, vol. 16, no. 2, April 2016. pp. 373-392. Project Muse,
http://muse.jhu.edu/article/613847. Accessed 29 June 2017.
Selected References

23. 23
Comments or Questions?
THANK YOU!
Contacts:
Joleen McInnis, jwesterd@odu.edu
Lucinda Rush, lrush@odu.edu
Leo S. Lo, llo@odu.edu