Academic libraries need new methods of demonstrating their value to their institutions and their patrons. One potential method is to investigate a link between library usage and student attainment. This presentation describes some work undertaken in DBS library looking at the possible effect of a number of library usage criteria on the final exam grade achieved by final year degree students. A correlation was found between final exam grade library borrowing, off-campus resource usage & printing from Moodle and final exam grade. In addition, mandatory attendance at a information skills class increased library borrowing and off-campus electronic resource usage. This leads to the supposition that information literacy instruction may drive better exam results.

1. Measuring Library Impact at
DBS
David Hughes
Systems Librarian
Dublin Business School

2. “You can observe a lot by just watching”
Yogi Berra
“Aw, people can come up with statistics to
prove anything, Kent. Forfty [sic] percent of all
people know that.”
Homer Simpson

3. http://www.artscouncil.org.uk/media/uploads/pdf/Evidence_review_economi
c_contribution_libraries.pdf

4. Question
Does the library make a difference to
student attainment?

5. Why?
The library does not exist in a vacuum
Traditional measurements of library ‘value’ not
so appropriate these days

6. Why: Culture
We live in an era of quantitative assessment (i.e.
an obsession with measurement)

7. Why:The Next Big Thing
http://library.wiki.nmc.org/Learning+Analytics

8. Why:Validation
Can help reassure that we are providing a
good (i.e. are we
doing the right
things) service to
our users

9. Why: Marketing
Use the library and get more smart, duh.

10. Antecedents
University ofWollongong Library: the
Library Cube
Tailored database and reporting
function joining library usage data
with student data
Found a strong correlation between
students' grades and use of
information resources the library
provides.
http://www.educause.edu/ero/article/discovering-impact-library-use-and-student-
performance

11. Antecedents
JISC Library Impact Project:
Aimed to prove a statistically significant
correlation between library usage and
student attainment at 8 universities.
https://library3.hud.ac.uk/blogs/lidp/

12. JISC Library Impact Project
E-resources usage, library borrowing
statistics, and library footfall
“The research successfully demonstrated a
statistically significant relationship between
library resource use and level of degree
result”
Stone, G., & Ramsden, B. (2013). Library Impact Data Project: Looking for
the Link between Library Usage and Student Attainment. College &
Research Libraries, 74(6), 546–559. http://doi.org/10.5860/crl12-406

13. So why bother?
Reproducibility
“What Kabir talks of is only what he
has lived through.
If you have not lived through something
it is not true.”
Kabir
“..Recent studies indicate that up to
70% of research from academic labs
cannot be reproduced, representing an
enormous waste of money and effort”
Dr. Elizabeth Iorns CEO Science
Exchange
http://blog.scienceexchange.com/2012/
08/the-reproducibility-initiative/

14. Caveat: Lots of limitations
Me vs 16+ Library Impact project members
Lack of statistical analysis
Access to data (e.g. library footfall, on-
campus electronic resource access)
Student demographics/external factors
Crude measurements

15. Looked at
Final exam grade and
Library borrowing
Off-campus resource usage
Attending a library class
Printing (Yes, printing)

16. Materials and Methods 1
Library Management System
(Heritage, Koha)
Circulation Reports
Patron Attributes
Slave labour
FinalYear Students
Open Athens (electronic resource access management)
Papercut (Print Management System)
Student Management System
(Prestige,Agresso)
MS Excel
MS Access

17. Materials + Methods 2
Run circulation/usage/attendance reports
Run exam grade report
Export reports to Excel
Import to Access
Run Access queries

18. Materials + Methods 3

19. Library Borrowing 2009-2010
0
10
20
30
40
50
60
70
1st Class Honours Upper Second Lower Second 3rd Class
Library Usage and Exam Success 2009-2010
Average Library Usage

20. Library Borrowing 2010-2011
0
10
20
30
40
50
60
FIRST CLASS HONOURS
DEGREE
UPPER SECOND CLASS
HONOURS DEGREE
LOWER SECOND CLASS
HONOURS DEGREE
THIRD CLASS
HONOURS DEGREE
Library Usage and Exam Success 2010-2011
Average Library Usage

21. Library Borrowing 2011-2012
0
10
20
30
40
50
60
FIRST CLASS HONOURS UPPER SECOND CLASS
HONOURS
LOWER SECOND CLASS
HONOURS
THIRD CLASS
HONOURS
Library Usage and Exam Success 2011-2012
Average Library Usage

22. Library Borrowing 2012-2013

23. Library Borrowing 2013-2014
0
10
20
30
40
50
60
70
80
90
1st Class Honours Upper 2nd Class Honours Lower 2nd Class Honours Third Class Honours
Library usage and exam success 2013-2014
Average borrowing per student

24. Cumulatively
0
10
20
30
40
50
60
70
1st Class Honours Upper Second Lower Second 3rd Class
Library Borrowing and Exam Grade 2009/2014
Average Borrowing

25. By counts, not average
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
Non-users 1 to 20 books 21+ books
Library Borrowing and Exam Success 2013-2014
H1
H21
H22
H3

26. However…

27. Mexican Lemon Imports Prevent US
Highway Deaths
http://scienceblogs.com/commonknowledge/20
09/04/01/mexican-lemons-prevent-highway/

28. But
Would we really, REALLY expect to see
this?
0
10
20
30
40
50
60
70
FIRST CLASS
HONOURS
UPPER SECOND
CLASS
HONOURS
LOWER
SECOND CLASS
HONOURS
THIRD CLASS
HONOURS
Fantasy Graph
Average Borrowing

29. Student Engagement
Engaged students borrow more from the
library
Engaged students get better grades
Increased borrowing = better grades???????

30. Off-campus resource usage
0
5
10
15
20
25
30
1st Class Honours Upper 2nd Class
Honours
Lower 2nd Class
Honours
3rd Class Honours
Athens usage and exam success 2013-2014
Average Usage

31. Off-campus resource usage
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
1st Class Honours Upper 2nd Class
Honours
Lower 2nd Class
Honours
3rd Class Honours
Athens Use and Exam Grade
Users
Non-users

32. Student engagement (2)
Engaged students access electronic
resources more
Engaged students get better grades
More electronic resource logins = better
grades???????

33. Library Classes
Drop-in/Sign-Up classes on e.g.,
Referencing
Zotero
Database Searching
Embedded classes
Part of module
Integrated classes
invited by lecturer
Do they have any impact?

34. Yes! (1)
0.00
5.00
10.00
15.00
20.00
25.00
30.00
35.00
40.00
45.00
50.00
Non-Attendees Library Class Attendees
Library Borrowing vs Library Class Attendance
Average Borrowing

35. Yes! (2)
0
2
4
6
8
10
12
14
16
18
Non-Attendees Library Class Attendees
Average Usage

36. Student engagement (3)
Students who take drop-in library class are
engaged
Drop-in library classes encourage library
use
Engaged students make more use of the
library

37. Although
Attendance at (Embedded/Integrated)
classes is voluntary
Embedded = part of module
Integrated = invited by lecturing staff

38. More impact (1)
0
10
20
30
40
50
60
70
Not Attending Embedded/Integrated Sign-Up Classes
Library Borrowing vs Library Class Attendance
Average Borrowing

39. More Impact (2)
0
2
4
6
8
10
12
14
16
18
Not Attending Embedded/Integrated Sign-Up Classes
Athens Usage vs Library Class Attendance
Average usage

40. Conclusion: Library Classes…
will increase student borrowing and
off-campus resource [Athens] usage
Do they improve exam results?

41. Remember
“Aw, people can come up with statistics to
prove anything, Kent. Forfty [sic] percent of all
people know that.”
Homer Simpson

42. Library Classes and Exam Grade
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
1st Class Honours Upper 2nd Class Honours Lower 2nd Class Honours H3
Library Class Attendence and Exam Success
Non-Attending
Attending

43. Embedded Library Classes
“Learning to Learn”
Commenced September 2013
Level 8 Business students
Subsequently rolled out to other classes

44. Embedded Library Class Attendees
0.00%
10.00%
20.00%
30.00%
40.00%
50.00%
60.00%
70.00%
80.00%
90.00%
100.00%
Sit exams Score 70-
100% in
exams
Score 60-
69% in
exams
Pass exams
% of students
on course
Students on courses with learning to learn element are more likely to...
The effects of embedded library classes I
Learning to Learn
Non-Learning to Learn

45. Embedded Library Class Attendees
0.00%
5.00%
10.00%
15.00%
20.00%
25.00%
30.00%
35.00%
40.00%
45.00%
Score 50-59%
in exams
Score 40-49%
in exams
Fail exams Be non-active
on course
% of students
on course
Students on courses with learning to learn element are less likely to...
The effects of embedded library classes II
Learning to Learn
Non-Learning to Learn

46. https://xkcd.com/552/

47. If nothing else
Advocacy & Marketing
We have some nice slides for
advocacy/marketing
purposes

48. Printing
Free student printing from Moodle
6 (b/w) or 25 (colour) cents/sheet for
other printing

49. The key to exam success … print
more…
0
200
400
600
800
1000
1200
1400
1600
1800
H1 H21 H22 H3
Pages Printed
Honours Class
Printing and Exam Success 2013-2014
Average Pages Printed

50. … but don’t pay for it
0
10
20
30
40
50
60
70
H1 H21 H22 H3
Honours Class
Print spending and Exam Success 2013-2014
Money spent (Euro)

51. Tentative conclusions/hypotheses
The library has the right books and right
electronic resources
Library classes might improve student
engagement and so their final exam grade
Link between free printing and exam grade
Moodle notes are effective
Students prefer reading print to reading on
screen (or information retained better
when read in print)

52. Next
More data - more library class analysis
Statistical analysis
Library measurement as a tool not to
measure library impact, but measure and
monitor student engagement  student
progression?
Communication/collaboration
with other departments?
with other libraries?

53. Thank you for listening