About the Webinar
In a time of shrinking budgets and growing reliance on electronic resources, the collection and analysis of usage statistics has become a staple of the library world. But while usage statistics may be ubiquitous, many librarians still struggle with the best methods of interpreting the data. The ability to effectively understand and apply usage data is an important skill for librarians to master as they attempt to analyze their collections and justify their expenses to administrations.
This webinar will highlight the ins and outs of COUNTER, as well as discuss the process of analyzing the data once harvested.Introductions
Agenda
Todd Carpenter, Executive Director, NISO
Todd Enoch, Head, Serials and Electronic Resources, University of North Texas Libraries;
Chair of the Continuing Education Committee, NASIG
* * * * * * *
COUNTER Update: Release 4 of the COUNTER Code of Practice for e-Resources
Peter Shepherd, Project Director, COUNTER
Integrating COUNTER Statistics within the Information Workflow
Oliver Pesch, Chief Product Strategist and Senior Vice President, EBSCO Information Services
Usage in the Eye of the Beholder: Developing Academic Library Usage Reports that Meet the Needs of Your Institution
Jill Emery, Collection Development Librarian, Portland State University Library
Student Profile Sample - We help schools to connect the data they have, with ...
May 21 NISO/NASIG Webinar: Playing the Numbers: Best Practices in Acquiring, Interpreting and Applying Usage Statistics
1. NISO/NASIG Joint Webinar:
Playing the Numbers:
Best Practices in Acquiring, Interpreting and
Applying Usage Statistics
May 21, 2014
Speakers:
Peter Shepherd, Project Director,
COUNTER
Oliver Pesch, Chief Product Strategist and Senior Vice President,
EBSCO Information Services
Jill Emery, Collection Development Librarian,
Portland State University Library
http://www.niso.org/news/events/2014/nasig/
2. Release 4 of the COUNTER
Code of Practice for e-
Resources and new usage-
based measures of impact
Peter Shepherd
COUNTER
May 2014
3. COUNTER Release 4
- objectives
A single, unified Code covering all e-resources,
including journals, databases, books, reference works,
multimedia content, etc.
Improve the database reports
Improve the reporting of archive usage
Enable the reporting of mobile usage separately
Expand the categories of ‘Access Denied’ covered
Improve the application of XML and SUSHI in the
design of the usage reports
Collect metadata that facilitates the linking of usage
statistics to other datasets, such as subscription
information
4. Release 4: main features
A single, integrated Code of Practice covering
journals, databases, books, reference works
and multimedia content
An expanded list of Definitions, including terms
such as ‘Gold Open Access’, ‘Multimedia Full
Content Unit’, ‘Record View’, ‘Result Click’, as
well as different categories of ‘Access Denied’,
etc. that are used for the first time in Release 4
Enhancements of the SUSHI (Standardised
Usage Statistics Harvesting Initiative) protocol
designed to facilitate its implementation by
vendors and its use by librarians
5. Release 4: main features
A requirement that Institutional Identifiers, Journal DOI
and Book DOI be included in the usage reports, to
facilitate not only the management of usage data, but
also the linking of usage data to other data relevant to
collections of online content.
A requirement that usage of Gold Open Access articles
within journals be reported separately in a new report:
Journal Report 1 GOA: Number of Successful Gold Open
Access Full-text Article Requests by Month and Journal.
A requirement that Journal Report 5 must be provided
6. Release 4: main features
Modified Database Reports, in which the previous requirement to report
Session counts has been dropped, and new requirements, to report
Record Views and Result Clicks, have been added. (Database Report 3
has also been renamed Platform Report 1).
A new report, Multimedia Report 1, which covers the usage of non-
textual multimedia resources, such as audio, video and images, by
reporting the number of successful requests for multimedia full content
units
New optional reports covering usage on mobile devices
A description of the relative advantages of logfiles and page tags as the
basis for counting online usage
Flexibility in the usage reporting period that allows customers to specify
a date range for their usage reports
7. Release 4: Standard Usage
Reports
Journal Report 1: Number of Successful Full-Text Article Requests by Month and
Journal
Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article
Requests by Month and Journal
Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and
Category
Journal Report 5: Number of Successful Full-Text Article Requests by Year-of-
Publication (YOP) and Journal
Database Report 1: Total Searches, Result Clicks and Record Views by Month
and Database
Database Report 2: Access Denied by Month, Database and Category
Platform Report 1: Total Searches, Result Clicks and Record Views by Month and
Platform
Book Report 1: Number of Successful Requests by Month and Title
Book Report 2: Number of Successful Section Requests by Month and Title
Book Report 3: Access Denied to Content Items by Month, Title and Category
Book Report 4: Access Denied to Content Items by Month, Platform and Category
Book Report 5: Total Searches by Month and Title
Multimedia Report 1: Number of Successful Full Multimedia Content Units
Requests by Month and Collection
8. Release 4: recording and reporting
usage on mobile devices
The following optional additional reports enable usage on mobile devices to be
reported separately:
Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal
and Page Type for usage on a Mobile Device
Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles
and Book Sections by Month and Title ( formatted for normal browsers/delivered to
mobile devices AND formatted for mobile devices/delivered to mobile devices)
Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page
Type (formatted for normal browsers/delivered to mobile devices AND formatted
for mobile devices/delivered to mobile devices)
COUNTER will recognize as usage on a mobile device, which may be reported in the
above reports, any usage that meets one of the following criteria:
useragents that are included in the WURFL list. WURFL is the Wireless
Universal Resource FiLe, a database containing the profile of mobile devices; this
database may be found at: http://wurfl.sourceforge.net/
usage via a proprietary mobile App provided by the publisher/content provider
9. Release 4: timetable for
implementation
Deadline date for implementation of Release 4:
31 December 2013
-after this date only vendors compliant with Release
4 are COUNTER compliant
Over 100 Publishers/Vendors are now providing Release
4 Usage Reports
10. Full details of Release 4 will be found on the
COUNTER website at:
http://www.projectcounter.org/code_practice.html
COUNTER Code of Practice
-Release 4
11. New COUNTER usage-based
measures of impact
Advantages:
Usage can be reported at the individual item and individual researcher level
Usage is more ’immediate’ than citations
Usage potentially covers all categories of online publication
COUNTER usage statistics are independently audited and generally trusted
Two new COUNTER Codes of Practice have been launched:
COUNTER Code of Practice for Articles (COUNTER Articles)
Recording, consolidation and reporting of usage at the individual article level
Standard applies to publishers, aggregators and repositories
COUNTER Code of Practice for Usage Factor
Usage-based measure of impact of journals, institutions and individual scholars
The Usage Factor for a Journal is the Median Value in a set of ordered full-text article usage data ( i.e. the
number of successful full text article requests) for a specified Usage Period of articles published in a journal
during a specified Publication Period.
COUNTER Articles and Usage Factor are both based on the recording and
consolidation of COUNTER-compliant usage data at the individual article level
12. COUNTER Code of Practice for
Articles
COUNTER Articles covers the following areas:
article types to be counted;
article versions to be counted;
data elements to be measured;
definitions of these data elements;
content and format of usage reports;
requirements for data collection and data processing;
requirements for independent audit (under development);
Release 1 of the COUNTER Code of Practice for Articles is available on the
COUNTER website at: http://www.projectcounter.org/counterarticles.html
13. Usage Factor: aims and outcomes
The overall aim of the Usage Factor project was to explore how online journal usage
statistics might form the basis of a new measure of journal impact and quality, the
Usage Factor for journals.
Specific objectives were to answer the following questions:
Will Usage Factor be a statistically meaningful measure?
Will Usage Factor be accepted by researchers, publishers, librarians and research
institutions?
Will Usage Factor be statistically credible and robust?
Is there an organizational and economic model for its implementation that would
cost-effective and be acceptable to the major stakeholder groups.
Following extensive testing using usage data for over 200 journals from a range of
publishers the main outcome of the project has been the new COUNTER Code of
Practice for Usage Factors. This new Code of Practice uses the article level usage
data collected using the COUNTER Code of Practice for Articles as the basis for
the calculation of the Usage Factor.
The COUNTER Code of Practice for Usage Factors is available on the COUNTER
website at: http://www.projectcounter.org/usage_factor.html
14. Who will benefit from the Usage
Factor?
Four major groups will benefit from the introduction of Usage Factors:
Authors, especially those in practitioner-oriented fields, where citation-
based measures understate the impact of journals, as well as those in
areas outside the core STM fields of pure research, where coverage of
journals by citation-based measures is weak.
Publishers, especially those with large numbers of journals outside of the
core STM research areas, where there is no reliable, universal measure of
journal impact, because citation-based measures are either inadequate or
non-existent for these fields
Librarians, when deciding on new journal acquisitions, have no reliable,
global measures of journal impact for fields outside the core STM
research fields. They would use usage-based measures to help them
prioritise journals to be added to their collections.
Research Funding Agencies, who are seeking a wider range of credible,
consistent quantitative measures of the value and impact of the outputs of
the research that they fund.
15. Usage Factor: Journals
- the calculation
Publishers will be able to generate Usage Factors using the Code of Practice, but will
have to be independently audited for their Usage Factors to be listed in the Usage
Factor Central Registry. Two categories of Usage Factor may be calculated
The 24 month Journal Usage Factor 2010/2011: all content
The median number of successful requests during 2010/2011 to content
published in the journal in 2010/2011
The Journal Usage Factor 2010/2011: full-text articles only
The median number of successful requests during 2010/2011 to full-text
articles published in the journal in 2010/2011
Note:
1.The article-level data collected in COUNTER Article Report 1 will be used as
the basis for the Usage Factor calculation
2. Usage Factors will be reported annually, for 2010/2011, 2011/2012, etc.
16. COUNTER Articles and Usage
Factor - implementation
Step 1: implement COUNTER Code of Practice
for Articles
Step 2: Collect article-level usage data for
2014/2015
Step 3: Calculate and report Usage Factors
using protocols specified in Code of Practice for
Usage Factors
17. Integrating COUNTER Statistics within the
Information Workflow
Oliver Pesch
Chief Product Strategist
EBSCO Information Services
20. Effective collection analysis involves
combining usage with other data for a
more complete picture
and presenting that information in a
meaningful context.
33. Usage: The Key Metric
% of librarians indicating a metric as very important when making
content decisions
Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey, 2012
34. We asked librarians how they use
usage statistics:
98% for cancellation and/or renewal
decisions
72% for journal package negotiations
Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey,
2012
Why Usage Statistics Are Important
39. Usage from all platforms vs publisher platform
This reports provides
analysis for the
publisher platform and
all platforms.
Is this an important title
for this library’s users?
46. Putting it all together: the basics
Usage Cost
How do you
connect usage
to cost? ISSN?
47. Putting it all together: the basics
Usage Cost
How do you
connect usage
to cost? ISSN?
Challenges with ISSN
matching
- Wrong or missing ISSN
- Same ISSN on multiple rows
in COUNTER reports
- Same ISSN on multiple
reports
- Combinations have one cost
item but multiple usage entries
- Titles change publishers
(ISSN changes)
- Publishers change platforms
48. Putting it all together: packages
Usage Cost
The goal
Create a package analysis
with package-level summary.
49. Putting it all together: packages
Usage Cost
How do you
know what is
included in the
package?
Challenges
Analyzing the package as a
unit means accumulating
cost and usage for the entire
package.
COUNTER statistics are
reported at the “Platform”
level so the package content
is not defined in the report.
50. Putting it all together: packages
Usage Cost
How do you
know what is
included in the
package?Solution
Incorporate detailed
holdings and/or package
definition into the analysis.
Holdings
51. Putting it all together: other data
Usage CostHoldings Other
The goal
Add impact factors and other
key-decision data to the
report.
52. Putting it all together: other data
Usage Cost
How do you
connect the other
data? ISSN?
Holdings Other
53. Putting it all together: other data
Usage Cost
How do you
connect the other
data? ISSN?
Holdings Other
Inclusion of impact
factors, faculty
recommendations, do-
not-cancel information
has similar mapping
challenges.
54. Putting it all together
Usage CostHoldings Other
The goal
Make this process
repeatable and as simple as
possible.
55. Putting it all together
Usage CostHoldings Other
What technology can
do all this?
56. Putting it all together
Usage CostHoldings Other
What technology can
do all this?
Some options
• Excel
• MS Access or similar
• Commercial products
with central knowledge
base to handle the
mappings.
57. Summary
• Usage is a means to an end
• Usage needs to be combined with other data to be truly
effective
• Providing usage and related metrics within the workflow
can improve efficiency and quality of decision making
• The act of “combining” usage and other data has several
challenges
• Close sometimes has to be good enough
• Knowledge base centric applications will help with the
mapping
59. Usage in the
Eye of the
Beholder:
Developing Academic
Library Usage Reports
that Meet the Needs of
Your Institution
Jill Emery
Collection Development
Librarian
Portland State University
61. Determine the audience
Different
audiences
require different
data
Understand
level of detail
sought
LSE, Audience in the Old Theatre, c1981
62. What is to Be Answered?
Knowing the
outcomes
sought upfront,
saves time with
data gathering
Review
outcomes with
vested parties
63. Less is More
Curtail data
Customize level
of detail
Be as concise as
possible OSU Special Collections & Archives: Crater
Lake, looking north from the Lodge
67. Multi-Media Reports
New with
Counter 4 stats
Advocate for
providers to
become
compliant
U.S. National Archives: Nitrate Film Stored
in a Brick Vault
68. Gold Open Access
Another new
report with
Counter 4
standard report
Can be used in
price
negotiation
72. Purchasing Reports
More granular
detail
May be multi-
year
May need to
adjust more
often
Field Museum, Page 74 of Accounting
Administration from Financial Ledger from
World's Columbian Exposition 1893
76. Administrative
Keep to a page
or short slide
deck (5 slides)
Use graphs/pie
charts to convey
salient
information
Smithsonian, Mary Blade standing at
the blackboard
78. Collection Decision
Reports
Almost always
spreadsheets
Simplified from
COUNTER
reports
Contextual
detail California Historical Society: Menu, Ritz Old
Poodle Dog, San Francisco
80. NISO/NASIG Joint Webinar • May 21, 2014
Questions?
All questions will be posted with presenter answers on
the NISO website following the webinar:
http://www.niso.org/news/events/2014/nasig/
NISO/NASIG Webinar:
Playing the Numbers: Best Practices in Acquiring,
Interpreting and Applying Usage Statistics
81. Thank you for joining us today.
Please take a moment to fill out the brief online survey.
We look forward to hearing from you!
THANK YOU