May 21 NISO/NASIG Webinar: Playing the Numbers: Best Practices in Acquiring, Interpreting and Applying Usage Statistics

Like this? Share it with your network

Share

May 21 NISO/NASIG Webinar: Playing the Numbers: Best Practices in Acquiring, Interpreting and Applying Usage Statistics

  • 1,484 views
Uploaded on

...


About the Webinar

In a time of shrinking budgets and growing reliance on electronic resources, the collection and analysis of usage statistics has become a staple of the library world. But while usage statistics may be ubiquitous, many librarians still struggle with the best methods of interpreting the data. The ability to effectively understand and apply usage data is an important skill for librarians to master as they attempt to analyze their collections and justify their expenses to administrations.

This webinar will highlight the ins and outs of COUNTER, as well as discuss the process of analyzing the data once harvested.Introductions

Agenda

Todd Carpenter, Executive Director, NISO

Todd Enoch, Head, Serials and Electronic Resources, University of North Texas Libraries;
Chair of the Continuing Education Committee, NASIG

* * * * * * *

COUNTER Update: Release 4 of the COUNTER Code of Practice for e-Resources
Peter Shepherd, Project Director, COUNTER

Integrating COUNTER Statistics within the Information Workflow
Oliver Pesch, Chief Product Strategist and Senior Vice President, EBSCO Information Services


Usage in the Eye of the Beholder: Developing Academic Library Usage Reports that Meet the Needs of Your Institution
Jill Emery, Collection Development Librarian, Portland State University Library

More in: Education , Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,484
On Slideshare
881
From Embeds
603
Number of Embeds
2

Actions

Shares
Downloads
29
Comments
0
Likes
1

Embeds 603

http://www.niso.org 602
http://news.google.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. NISO/NASIG Joint Webinar: Playing the Numbers: Best Practices in Acquiring, Interpreting and Applying Usage Statistics May 21, 2014 Speakers: Peter Shepherd, Project Director, COUNTER Oliver Pesch, Chief Product Strategist and Senior Vice President, EBSCO Information Services Jill Emery, Collection Development Librarian, Portland State University Library http://www.niso.org/news/events/2014/nasig/
  • 2. Release 4 of the COUNTER Code of Practice for e- Resources and new usage- based measures of impact Peter Shepherd COUNTER May 2014
  • 3. COUNTER Release 4 - objectives  A single, unified Code covering all e-resources, including journals, databases, books, reference works, multimedia content, etc.  Improve the database reports  Improve the reporting of archive usage  Enable the reporting of mobile usage separately  Expand the categories of ‘Access Denied’ covered  Improve the application of XML and SUSHI in the design of the usage reports  Collect metadata that facilitates the linking of usage statistics to other datasets, such as subscription information
  • 4. Release 4: main features  A single, integrated Code of Practice covering journals, databases, books, reference works and multimedia content  An expanded list of Definitions, including terms such as ‘Gold Open Access’, ‘Multimedia Full Content Unit’, ‘Record View’, ‘Result Click’, as well as different categories of ‘Access Denied’, etc. that are used for the first time in Release 4  Enhancements of the SUSHI (Standardised Usage Statistics Harvesting Initiative) protocol designed to facilitate its implementation by vendors and its use by librarians
  • 5. Release 4: main features  A requirement that Institutional Identifiers, Journal DOI and Book DOI be included in the usage reports, to facilitate not only the management of usage data, but also the linking of usage data to other data relevant to collections of online content.  A requirement that usage of Gold Open Access articles within journals be reported separately in a new report: Journal Report 1 GOA: Number of Successful Gold Open Access Full-text Article Requests by Month and Journal.  A requirement that Journal Report 5 must be provided
  • 6. Release 4: main features  Modified Database Reports, in which the previous requirement to report Session counts has been dropped, and new requirements, to report Record Views and Result Clicks, have been added. (Database Report 3 has also been renamed Platform Report 1).  A new report, Multimedia Report 1, which covers the usage of non- textual multimedia resources, such as audio, video and images, by reporting the number of successful requests for multimedia full content units  New optional reports covering usage on mobile devices  A description of the relative advantages of logfiles and page tags as the basis for counting online usage  Flexibility in the usage reporting period that allows customers to specify a date range for their usage reports
  • 7. Release 4: Standard Usage Reports  Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal  Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal  Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and Category  Journal Report 5: Number of Successful Full-Text Article Requests by Year-of- Publication (YOP) and Journal  Database Report 1: Total Searches, Result Clicks and Record Views by Month and Database  Database Report 2: Access Denied by Month, Database and Category  Platform Report 1: Total Searches, Result Clicks and Record Views by Month and Platform  Book Report 1: Number of Successful Requests by Month and Title  Book Report 2: Number of Successful Section Requests by Month and Title  Book Report 3: Access Denied to Content Items by Month, Title and Category  Book Report 4: Access Denied to Content Items by Month, Platform and Category  Book Report 5: Total Searches by Month and Title  Multimedia Report 1: Number of Successful Full Multimedia Content Units Requests by Month and Collection
  • 8. Release 4: recording and reporting usage on mobile devices  The following optional additional reports enable usage on mobile devices to be reported separately:  Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal and Page Type for usage on a Mobile Device  Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles and Book Sections by Month and Title ( formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)  Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page Type (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices) COUNTER will recognize as usage on a mobile device, which may be reported in the above reports, any usage that meets one of the following criteria:  useragents that are included in the WURFL list. WURFL is the Wireless Universal Resource FiLe, a database containing the profile of mobile devices; this database may be found at: http://wurfl.sourceforge.net/  usage via a proprietary mobile App provided by the publisher/content provider
  • 9. Release 4: timetable for implementation  Deadline date for implementation of Release 4: 31 December 2013 -after this date only vendors compliant with Release 4 are COUNTER compliant  Over 100 Publishers/Vendors are now providing Release 4 Usage Reports
  • 10. Full details of Release 4 will be found on the COUNTER website at: http://www.projectcounter.org/code_practice.html COUNTER Code of Practice -Release 4
  • 11. New COUNTER usage-based measures of impact Advantages:  Usage can be reported at the individual item and individual researcher level  Usage is more ’immediate’ than citations  Usage potentially covers all categories of online publication  COUNTER usage statistics are independently audited and generally trusted Two new COUNTER Codes of Practice have been launched: COUNTER Code of Practice for Articles (COUNTER Articles)  Recording, consolidation and reporting of usage at the individual article level  Standard applies to publishers, aggregators and repositories COUNTER Code of Practice for Usage Factor  Usage-based measure of impact of journals, institutions and individual scholars  The Usage Factor for a Journal is the Median Value in a set of ordered full-text article usage data ( i.e. the number of successful full text article requests) for a specified Usage Period of articles published in a journal during a specified Publication Period. COUNTER Articles and Usage Factor are both based on the recording and consolidation of COUNTER-compliant usage data at the individual article level
  • 12. COUNTER Code of Practice for Articles COUNTER Articles covers the following areas:  article types to be counted;  article versions to be counted;  data elements to be measured;  definitions of these data elements;  content and format of usage reports;  requirements for data collection and data processing;  requirements for independent audit (under development); Release 1 of the COUNTER Code of Practice for Articles is available on the COUNTER website at: http://www.projectcounter.org/counterarticles.html
  • 13. Usage Factor: aims and outcomes The overall aim of the Usage Factor project was to explore how online journal usage statistics might form the basis of a new measure of journal impact and quality, the Usage Factor for journals. Specific objectives were to answer the following questions:  Will Usage Factor be a statistically meaningful measure?  Will Usage Factor be accepted by researchers, publishers, librarians and research institutions?  Will Usage Factor be statistically credible and robust?  Is there an organizational and economic model for its implementation that would cost-effective and be acceptable to the major stakeholder groups. Following extensive testing using usage data for over 200 journals from a range of publishers the main outcome of the project has been the new COUNTER Code of Practice for Usage Factors. This new Code of Practice uses the article level usage data collected using the COUNTER Code of Practice for Articles as the basis for the calculation of the Usage Factor. The COUNTER Code of Practice for Usage Factors is available on the COUNTER website at: http://www.projectcounter.org/usage_factor.html
  • 14. Who will benefit from the Usage Factor? Four major groups will benefit from the introduction of Usage Factors:  Authors, especially those in practitioner-oriented fields, where citation- based measures understate the impact of journals, as well as those in areas outside the core STM fields of pure research, where coverage of journals by citation-based measures is weak.  Publishers, especially those with large numbers of journals outside of the core STM research areas, where there is no reliable, universal measure of journal impact, because citation-based measures are either inadequate or non-existent for these fields  Librarians, when deciding on new journal acquisitions, have no reliable, global measures of journal impact for fields outside the core STM research fields. They would use usage-based measures to help them prioritise journals to be added to their collections.  Research Funding Agencies, who are seeking a wider range of credible, consistent quantitative measures of the value and impact of the outputs of the research that they fund.
  • 15. Usage Factor: Journals - the calculation Publishers will be able to generate Usage Factors using the Code of Practice, but will have to be independently audited for their Usage Factors to be listed in the Usage Factor Central Registry. Two categories of Usage Factor may be calculated  The 24 month Journal Usage Factor 2010/2011: all content The median number of successful requests during 2010/2011 to content published in the journal in 2010/2011  The Journal Usage Factor 2010/2011: full-text articles only The median number of successful requests during 2010/2011 to full-text articles published in the journal in 2010/2011  Note: 1.The article-level data collected in COUNTER Article Report 1 will be used as the basis for the Usage Factor calculation 2. Usage Factors will be reported annually, for 2010/2011, 2011/2012, etc.
  • 16. COUNTER Articles and Usage Factor - implementation  Step 1: implement COUNTER Code of Practice for Articles  Step 2: Collect article-level usage data for 2014/2015  Step 3: Calculate and report Usage Factors using protocols specified in Code of Practice for Usage Factors
  • 17. Integrating COUNTER Statistics within the Information Workflow Oliver Pesch Chief Product Strategist EBSCO Information Services
  • 18. Gathering usage is just the first step…
  • 19. and usage is just one piece of the puzzle.
  • 20. Effective collection analysis involves combining usage with other data for a more complete picture and presenting that information in a meaningful context.
  • 21. Examples of usage in the workflow (presenting usage in context)
  • 22. Subscription renewals
  • 23. Subscription renewals
  • 24. Subscription renewals Instant access to cost- per-use and holdings data streamlines the renewal process.
  • 25. Individual Titles
  • 26. Individual Titles Detailed analysis, holdings and usage available for individual orders.
  • 27. Publisher packages
  • 28. Publisher packages Package-level totals presented
  • 29. Publisher packages Quick charts to see the “health” of the package.
  • 30. Publisher Packages
  • 31. Publisher Packages Usage and cost-per-use in package details facilitates decisions on “swaps” and “drops”.
  • 32. Usage vs other metrics
  • 33. Usage: The Key Metric % of librarians indicating a metric as very important when making content decisions Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey, 2012
  • 34. We asked librarians how they use usage statistics: 98% for cancellation and/or renewal decisions 72% for journal package negotiations Source: EBSCO, 2012; EBSCO Library Collection and Budgeting Trends Survey, 2012 Why Usage Statistics Are Important
  • 35. Including other metric types
  • 36. Including other metric types This library has included Impact Factor and Eigenfactor scores.
  • 37. Decisions and challenges related to usage and collection analysis
  • 38. Usage from all platforms vs publisher platform
  • 39. Usage from all platforms vs publisher platform This reports provides analysis for the publisher platform and all platforms. Is this an important title for this library’s users?
  • 40. Subscriptions to “Combinations” We have a single order that is for two titles.
  • 41. Subscriptions to “Combinations” The order is mapped to both titles in holdings
  • 42. Subscriptions to “Combinations” Usage for this “order” accumulates usage from both titles so cost- per-use is meaningful.
  • 43. Mapping the data and analyzing packages…
  • 44. Putting it all together: the basics The goal A report showing subscribed titles with their cost, use and cost-per-use.
  • 45. Putting it all together: the basics Usage Cost
  • 46. Putting it all together: the basics Usage Cost How do you connect usage to cost? ISSN?
  • 47. Putting it all together: the basics Usage Cost How do you connect usage to cost? ISSN? Challenges with ISSN matching - Wrong or missing ISSN - Same ISSN on multiple rows in COUNTER reports - Same ISSN on multiple reports - Combinations have one cost item but multiple usage entries - Titles change publishers (ISSN changes) - Publishers change platforms
  • 48. Putting it all together: packages Usage Cost The goal Create a package analysis with package-level summary.
  • 49. Putting it all together: packages Usage Cost How do you know what is included in the package? Challenges Analyzing the package as a unit means accumulating cost and usage for the entire package. COUNTER statistics are reported at the “Platform” level so the package content is not defined in the report.
  • 50. Putting it all together: packages Usage Cost How do you know what is included in the package?Solution Incorporate detailed holdings and/or package definition into the analysis. Holdings
  • 51. Putting it all together: other data Usage CostHoldings Other The goal Add impact factors and other key-decision data to the report.
  • 52. Putting it all together: other data Usage Cost How do you connect the other data? ISSN? Holdings Other
  • 53. Putting it all together: other data Usage Cost How do you connect the other data? ISSN? Holdings Other Inclusion of impact factors, faculty recommendations, do- not-cancel information has similar mapping challenges.
  • 54. Putting it all together Usage CostHoldings Other The goal Make this process repeatable and as simple as possible.
  • 55. Putting it all together Usage CostHoldings Other What technology can do all this?
  • 56. Putting it all together Usage CostHoldings Other What technology can do all this? Some options • Excel • MS Access or similar • Commercial products with central knowledge base to handle the mappings.
  • 57. Summary • Usage is a means to an end • Usage needs to be combined with other data to be truly effective • Providing usage and related metrics within the workflow can improve efficiency and quality of decision making • The act of “combining” usage and other data has several challenges • Close sometimes has to be good enough • Knowledge base centric applications will help with the mapping
  • 58. Thank you opesch@ebsco.com
  • 59. Usage in the Eye of the Beholder: Developing Academic Library Usage Reports that Meet the Needs of Your Institution Jill Emery Collection Development Librarian Portland State University
  • 60. Usage is Contextual
  • 61. Determine the audience  Different audiences require different data  Understand level of detail sought LSE, Audience in the Old Theatre, c1981
  • 62. What is to Be Answered?  Knowing the outcomes sought upfront, saves time with data gathering  Review outcomes with vested parties
  • 63. Less is More  Curtail data  Customize level of detail  Be as concise as possible OSU Special Collections & Archives: Crater Lake, looking north from the Lodge
  • 64. Gathering Stats  Sushi  Set ingest for right reports  Next Gen ILS management
  • 65. JR1 & JR1a  JR1-standard  JR1a-optional  Dedupe between the two LSE, Statistics Machine Room, 1964
  • 66. Book Reports  Growing demand  BR1-standard  BR2-standard
  • 67. Multi-Media Reports  New with Counter 4 stats  Advocate for providers to become compliant U.S. National Archives: Nitrate Film Stored in a Brick Vault
  • 68. Gold Open Access  Another new report with Counter 4  standard report  Can be used in price negotiation
  • 69. Typical Report Requests  Academic: ACRL, ARL, State-wide  Accreditation  Annual  Purchasing  Collection Decision
  • 70. Academic Reports  High level  Gross numbers  Little detail
  • 71. Annual & Accreditation  High level  Multi-year comparisons  More detail
  • 72. Purchasing Reports  More granular detail  May be multi- year  May need to adjust more often Field Museum, Page 74 of Accounting Administration from Financial Ledger from World's Columbian Exposition 1893
  • 73. Collection Decision Reports  Most detailed  Look at use and qualitative factors  Multi-year
  • 74. Report Structure  Remember audience  Questions to be answered  Level of detail needed NASA, Differential Analyzer
  • 75. Graphs/Pie Charts  Use for annual reports  Accreditation reports  Purchasing reports
  • 76. Administrative  Keep to a page or short slide deck (5 slides)  Use graphs/pie charts to convey salient information Smithsonian, Mary Blade standing at the blackboard
  • 77. Purchasing Reports  Combination of graphs/charts & spreadsheets  Longer to provide greater detail
  • 78. Collection Decision Reports  Almost always spreadsheets  Simplified from COUNTER reports  Contextual detail California Historical Society: Menu, Ritz Old Poodle Dog, San Francisco
  • 79. Thank you! Jill Emery Collection Development Librarian Portland State University jemery@pdx.edu
  • 80. NISO/NASIG Joint Webinar • May 21, 2014 Questions? All questions will be posted with presenter answers on the NISO website following the webinar: http://www.niso.org/news/events/2014/nasig/ NISO/NASIG Webinar: Playing the Numbers: Best Practices in Acquiring, Interpreting and Applying Usage Statistics
  • 81. Thank you for joining us today. Please take a moment to fill out the brief online survey. We look forward to hearing from you! THANK YOU