Metrics vs peer review: Why metrics can (and should?) be applied in the Socia...Anne-Wil Harzing
Similar to Scholarly eBooks in the Humanities and Social Sciences: Longitudinal Assessment of Project Muse/UPCC eBooks at Emory, Harvard, and Yale (20)
Z Score,T Score, Percential Rank and Box Plot Graph
Scholarly eBooks in the Humanities and Social Sciences: Longitudinal Assessment of Project Muse/UPCC eBooks at Emory, Harvard, and Yale
1. Association of Research Libraries
Licensing Initiative
for
University Press eBooks
Charleston Conference, 2014
Celeste Feather, LYRASIS
2. Background
2011: ARL contracted with LYRASIS to negotiate
offers for university press ebooks to:
Establish a model for the e-book landscape by
shaping offerings to serve the unique needs of the
academic library world in cost, ILL, platform
design, internal markup and search structures,
and interoperability
Encourage partnerships between libraries and
university presses to address e-book economics
and licensing
4. Participation in MUSE Offers
through the ARL/LYRASIS License
15 libraries purchased entire MUSE collection
12 libraries purchased pub. year collections
25 libraries purchased subject collections
5. Titles Used To Date in MUSE/UPCC
2010-2013 eBook Collections
By Emory, Harvard, and Yale
Used by 3
Institutions
9% Used by 2
Institutions
21%
Used by 1
Institution
32%
Titles Not Used
38%
8232 Titles
6. Return on Investment
Libraries generally achieve savings over the
cost of purchasing individual titles at MUSE
after they use:
At least 15% of the titles in the entire MUSE
collection
At least 35% of the titles in an annual
collection
Less than 50% of the titles in subject
collections (varies depending on subject)
7. Chris Palazzolo, PhD
Head of Collections & Social Sciences
Woodruff Library
Emory University
Charleston Conference 2014
8. Emory and Project Muse
Purchase of 2010-2013 complete collections; funding through common
good funds (funds shared by all Emory libraries)
Supplements for 2010-2013 purchased in mid-2014 (not included in
current data analysis)
2014 move to a la carte purchasing through YBP
MARC records loaded into back-end (Aleph) and piped into DiscoverE
(discovery layer)
No change to existing approval plan; only around 60 titles were only
available to users in e-format
10. BR4 Reports (Turnaway)
Availability of this Counter4 Report only became available in January
2014
95 un-owned titles were clicked on and patrons turned away
Of these 95 titles, 35 were checked out during the same period (January-
August 2014), primarily titles pre-2005
Led to addition of single title firm ordering in YBP to meet demand
11. Data Sources & Challenges
Collection-use, institution-specific statistics from Project Muse
Circulation statistics from ILS (Aleph) matched on print ISBN
Call number information not included in Project Muse data; not extracted from
circulation figures, but could be used to disaggregate some of the results (see
Sarah’s report)
Data leads to lots of hypotheses, but often times not definitive confirmations
Did users utilize e-book when print book was not available, or do they just have a
preference for the e-book format?
Who is utilizing the print title vs the e-book? Further analysis could be done on the
print side, utilizing demographic data from the ILS, but not from the e-book angle
(Shibboleth implementation may change this).
12. Overall Trends
Use of Project Muse Titles, 2010-2013
0 500 1000 1500 2000 2500
print titles circulated
print format only
titles accessed e-only
both formats circulated
titles accessed e
titles accessed e both formats circulated titles accessed e-only print format only print titles circulated
Series2 25.31% 8.61% 16.58% 7.93% 16.53%
Series1 2148 731 1407 673 1403
13. Specific Format Trends
45.00%
40.00%
35.00%
30.00%
25.00%
20.00%
15.00%
10.00%
5.00%
0.00%
% of Collections, 2010-3, Accessed in Specific
Format
Titles Accessed E (2010-2013) Both Formats (2010-2013)
14. Use of Collections by Format
Use of Collections by Format, 2010-2013
Ecology and Evolution
U.S. Regional Studies'
Political Science and Policy Studies
Poetry, Fiction and Creative Non-…
Philosophy and Religion
Language, Literature, Linguistics
History
Higher Education
Global Cultural Studies
Film, Theater and Performing Arts
Archaeology and Anthropology
0.00% 5.00% 10.00% 15.00% 20.00% 25.00%
Print--All Print Format Only E-Only/Exclusively
15. Proportion Comparison of
Format Use
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
Proportion Comparison of Format Use by Collection, 2010-
2013
Ecology and Evolution
U.S. Regional Studies'
Political Science and Policy Studies
Poetry, Fiction and Creative Non-Fiction
Philosophy and Religion
Literature
History
Higher Education
Global Cultural Studies
Film, Theater and Performing Arts
Archaeiology and Anthropology
16. Project Muse
Ebooks@ Yale University Library
Sarah Tudesco – Assessment Librarian
17. Project Muse Titles @ Yale
Brief History of Muse at Yale
• Acquired MUSE e-book package in FY2012
• Continued to receive print titles on approval through FY2014
• Adjusted approval profiles to electronic preferred in FY2015
Electronic Only
32%
Print & Electronic
68%
18. Print UPCC Circulation
Frequency @ Yale
34% of UPCC titles held at Yale circulated at least once. Includes
circulation that occurred before acquiring MUSE ebook package.
Never Circulated
66%
Circulated
34%
21. Project Muse – Electronic Usage Frequency
24% of MUSE titles accessed at least once.
In FY2014 – Yale acquired some of the other packages offered by
MUSE. Those titles have only been available for a few months.
76%
Titles Never Accessed
Titles Accessed
24%
24. Project Muse by LC Group
0 5,000 10,000 15,000 20,000
7,517
2,198
16,081
2,576
0% 20% 40% 60% 80% 100%
26%
25%
22%
74%
75%
78%
15% 85%
Analysis of COUNTER data from
2012 through Sept. 2014.
Groups
• Humanities (B, C, D, E-F, M, N, P)
• Sciences (Q, R, S, T, U, V)
• Social Sciences (G, H, J, K, L)
• Other (A, Z, No LC Number)
Humanities
Social Sciences
Sciences
Other/Unknown
Titles Accessed
Titles Not Accessed
25. Project Muse: Total Titles by Classification
0 1,000 2,000 3,000 4,000 5,000 6,000 7,000
P - Language and Literature
H - Social Sciences
E-F - History: America and United States
B - Philosophy, Psychology, Religion
D - History (except America)
J - Political Science
G - Geography, Anthropology
Q - Science
L - Education
K - Law
R - Medicine
M - Music
T - Technology and Engineering
N - Fine Arts
Titles Accessed
Titles Not Accessed
26. Activity by LC Classification: Print & Project Muse
Section Reqeusts (BR2) Print Circulation
0 10,000 20,000
P - Language and Literature 18,376
B - Philosophy, Psychology, Religion 14,676
H - Social Sciences 13,470
E-F - History: America 11,187
D - History (except America) 10,559
J - Political Science 4,233
G - Geography, Anthropology 3,434
K - Law 3,005
R - Medicine
Q - Science
2,749
2,504
N - Fine Arts 2,244
L - Education 1,295
M - Music 1,268
T - Technology 1,204
0 5,000 10,000
P 2,851
B 2,224
H 1,675
E-F 1,775
D 1,174
577
497
112
413
250
145
137
322
394
J
G
K
R
Q
N
L
M
T
31. Project MUSE eBooks at
Harvard University
Implementation and Assessment
Katherine Leach
Librarian for Western Languages Collections
Western Languages Division, Widener Library
Charleston Conference | November 7, 2014
36. Intentional Duplication
in Electronic and Print
70.7% of Project MUSE held in print at Harvard
24%
20%
3%
23%
13%
3%
4%
10%
Breakdown as follow
History
Social Sciences
Music & Fine Arts
Literature
Science & Technology
Philosophy & Religion
Anthropology
Other
37. Disciplines used in MUSE
23 % of Project MUSE was used/accessed
25%
17%
6%
21%
14%
4%
12%
1%
History
Political Science and Policy Studies
Film, Theater, and Performing Arts
Literature
US Regional Studies, South
Philosophy and Religion
Global Cultural Studies
Other (less than 750 uses)
38. Circulation
Some Unexpected Results
20% 18% 20%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Anthropology
Philosophy & Religion
Science & Technology
Literature
Music & Fine Arts
Social Sciences
History
Project MUSE
Print 2013
Print 2014
14% 16% 14%
24% 21% 20%
39. The current issue…
‘Monographs are different…this transition will take thoughtfulness
and care’
Kevin Guthrie, ITHAKA
(Symposium on Sustainable Models for Print Storage
in 21st Century Libraries, October 2, 2014)
40. Next Steps
• Qualitative study / Survey
• Continue gathering statistics
41. Thank you
Katherine Leach
Western Languages Division
Harvard University Library
kleach@fas.harvard.edu
Editor's Notes
Many libraries that invest in discovery/loading MARC records realize this ROI within 3 years after the collection purchase.
For print – 100% ROI is achieved when at least 22% of titles are used in entire collection, 52% of titles used in annual collection, varies for subjects: 60%+