NISO Webinar: Making Better Decisions with Usage Statistics
 

Like this? Share it with your network

Share

NISO Webinar: Making Better Decisions with Usage Statistics

on

  • 1,804 views

As libraries move to become centers of digital collections, maintaining information on the usage of these collections is ever more critical. It's also essential to be able to maintain common measures ...

As libraries move to become centers of digital collections, maintaining information on the usage of these collections is ever more critical. It's also essential to be able to maintain common measures across heterogeneous collections, in order to be able to effectively analyze how the library's collection dollars are being spent. The Project COUNTER Code of Practice and the SUSHI protocol aid in this work. This session will explore the newly-published Release 4 of the COUNTER Code of Practice for e-Resources and highlight its use in conjunction with the SUSHI (Standardized Usage Statistics Harvesting Initiative) protocol in an active library environment.

Statistics

Views

Total Views
1,804
Views on SlideShare
1,295
Embed Views
509

Actions

Likes
0
Downloads
29
Comments
0

2 Embeds 509

http://www.niso.org 507
http://translate.googleusercontent.com 2

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Lets look at COUNTER from the perspective of someone providing a usage consolidation service. COUNTER is the whole reason such services can exist in a scalable way… Having consistent comparable reports is a must.Combine COUNTER with SUSHI and the potential is there for great time savings by having reports automatically retrieved and loaded.COUNTER also covers a broad range which means that simply by supporting COUNTER a fairly comprehensive (and useful) service can be created.
  • But… there are some challenges.We have run across several “compliant” vendors that have reports that do not really meet the Code of Practice. Let me walk you through an actual example (with the names removed)…
  • Here is a database report 1… From a human-readable perspective, this looks pretty good. However, it is not COUNTER compliant.
  • Note the missing data elements…. Forget about a usage consolidation system for the moment and just consider what would happen if you used Excel to filter by “Total sessions” – you would have no clue what the numbers were for.
  • So to get this report to work, the missing data must be added
  • But there is more… there are blank rows between entries… this is not part of the COUNTER CoP…
  • So a librarian has to remove the blank lines.
  • THEN they have a file that can load… I am told it takes about 30 minutes to fix each file from this content provider.
  • And there are more issues like this… some are subtle like listing the report name in cell A1 as “COUNTER Journal Report 1” -- it is supposed to be “Journal Report 1” or missing the Platform name in the Totals row… or adding extra comments…
  • And there are more issues like this… some are subtle like listing the report name in cell A1 as “COUNTER Journal Report 1” -- it is supposed to be “Journal Report 1” or missing the Platform name in the Totals row… or adding extra comments…

NISO Webinar: Making Better Decisions with Usage Statistics Presentation Transcript

  • 1. http://www.niso.org/news/events/2012/nisowebinars/usage_statistics/ Making Better Decisions with Usage Statistics June 13, 2012Speakers: Peter Shepherd, Oliver Pesch, and Amy Lynn Fry
  • 2. Release 4 of the COUNTERCode of Practice for e-ResourcesPeter ShepherdCOUNTERJune 2012
  • 3. COUNTER Release 4- objectives A single, unified Code covering all e-resources, including journals, databases, books, reference works, multimedia content, etc. Improve the database reports Improve the reporting of archive usage Enable the reporting of mobile usage separately Expand the categories of „Access Denied‟ covered Improve the application of XML and SUSHI in the design of the usage reports Collect metadata that facilitates the linking of usage statistics to other datasets, such as subscription information
  • 4. Release 4: main features A single, integrated Code of Practice covering journals, databases, books, reference works and multimedia content An expanded list of Definitions, including terms such as „Gold Open Access‟, „Multimedia Full Content Unit‟, „Record View‟, „Result Click‟, as well as different categories of „Access Denied‟, etc. that are used for the first time in Release 4 Enhancements of the SUSHI (Standardised Usage Statistics Harvesting Initiative) protocol designed to facilitate its implementation by vendors and its use by librarians
  • 5. Release 4: main features A requirement that Institutional Identifiers, Journal DOI and Book DOI be included in the usage reports, to facilitate not only the management of usage data, but also the linking of usage data to other data relevant to collections of online content. A requirement that usage of Gold Open Access articles within journals be reported separately in a new report: Journal Report 1 GOA: Number of Successful Gold Open Access Full-text Article Requests by Month and Journal. A requirement that Journal Report 5 must be provided
  • 6. Release 4: main featuresModified Database Reports, in which the previous requirement to reportSession counts has been dropped, and new requirements, to report RecordViews and Result Clicks, have been added. (Database Report 3 has alsobeen renamed Platform Report 1).A new report, Multimedia Report 1, which covers the usage of non-textualmultimedia resources, such as audio, video and images, by reporting thenumber of successful requests for multimedia full content unitsNew optional reports covering usage on mobile devicesA description of the relative advantages of logfiles and page tags as thebasis for counting online usageFlexibility in the usage reporting period that allows customers to specify adate range for their usage reports
  • 7. Release 4: Standard UsageReports Journal Report 1: Number of Successful Full-Text Article Requests by Month and Journal Journal Report 1 GOA: Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal Journal Report 2: Access Denied to Full-Text Articles by Month, Journal and Category Journal Report 5: Number of Successful Full-Text Article Requests by Year-of- Publication (YOP) and Journal Database Report 1: Total Searches, Result Clicks and Record Views by Month and Database Database Report 2: Access Denied by Month, Database and Category Platform Report 1: Total Searches, Result Clicks and Record Views by Month and Platform Book Report 1: Number of Successful Requests by Month and Title Book Report 2: Number of Successful Section Requests by Month and Title Book Report 3: Access Denied to Content Items by Month, Title and Category Book Report 4: Access Denied to Content Items by Month, Platform and Category Book Report 5: Total Searches by Month and Title Multimedia Report 1: Number of Successful Full Multimedia Content Units Requests by Month and Collection
  • 8. Release 4: Journal Report 5
  • 9. Release 4: Database Report 1
  • 10. Release 4: Multimedia Report 1
  • 11. Release 4: recording and reportingusage on mobile devices The following optional additional reports enable usage on mobile devices to be reported separately: Journal Report 3 Mobile: Number of Successful Item Requests by Month, Journal and Page Type for usage on a Mobile Device Title Report 1 Mobile: Number of Successful Requests for Journal Full-text Articles and Book Sections by Month and Title ( formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices) Title Report 3 Mobile: Number of Successful Requests by Month, Title and Page Type (formatted for normal browsers/delivered to mobile devices AND formatted for mobile devices/delivered to mobile devices)COUNTER will recognize as usage on a mobile device, which may be reported in the above reports, any usage that meets one of the following criteria: useragents that are included in the WURFL list. WURFL is the Wireless Universal Resource FiLe, a database containing the profile of mobile devices; this database may be found at: http://wurfl.sourceforge.net/ usage via a proprietary mobile App provided by the publisher/content provider
  • 12. Release 4: timetable forimplementationDeadline date for implementation of Release 4:31 December 2013 -after this date only vendors compliant with Release 4 will be COUNTER compliantBetween now and 31 December 2013, Release 4 and theexisting Releases of the Codes of Practice are valid
  • 13. Release 4: complianceprocess1.Agree Release 4 usage reports that are relevant topublisher content2.Review of reports by a COUNTER library test site3.Vendor signs Declaration of COUNTER compliance4.Vendor added to the Register of COUNTER-compliantvendors5.Independent audit must be carried out and passed within 6months of being added to the Register
  • 14. Release 4: independentauditThree aspects of the audit  Check report formats  Check data integrity  Check delivery processThree possible audit outcomesA Pass, in which case no further action is required by the publisher as a result of theaudit. In some cases the auditor may add Observations to the audit report, which aredesigned to help the vendor improve its COUNTER usage reports, but which areoutside the scope of the audit itself.A Qualified Pass, in which the auditor deems the publisher to have passed the audit,but where the auditor raises a Minor Issue requiring further action to maintainCOUNTER-compliant status. A Minor Issue does not affect the reported figures, but isone which should be resolved within 3 months of the audit to maintain COUNTER-compliant status. An example of a Minor Issue is where a report format does notconform to the COUNTER specifications.A Fail, where the auditor has identified an issue that must be resolved immediatelyfor the vendor to maintain COUNTER -compliant status
  • 15. COUNTER Code of Practice-Release 4Full details of Release 4 will be found on theCOUNTER website at:http://www.projectcounter.org/code_practice.html
  • 16. Quality Content • Resource Management • Access • Integration • ConsultationMaking Better Decisions with Usage Statistics SUSHI: More relevant than ever Oliver Pesch Chief Strategist, E-Resource Access and Management Services
  • 17. Overview • Why SUSHI • COUNTER Release 4 • Changes to schemas • Procedural changes • Applying lessons learned • Other tools and resources available
  • 18. Why SUSHI… • Librarians doing more with less rely on usage statistics as one measure of value of their purchases • Usage consolidation applications and related services rely on COUNTER • To be valuable, usage collection needs to be comprehensive • Efficiency depends on automation • SUSHI is a very scalable standard for harvesting COUNTER reports!
  • 19. COUNTER 4: Schema Changes… SUSHI Schema (the actual SUSHI standard)
  • 20. COUNTER 4: Schema Changes… SUSHI Schema (the actual SUSHI standard) No changes were required!
  • 21. COUNTER 4: Schema Changes… COUNTER XML Schema for reports: changes were minor and geared towards better compliance, including: • ItemIdentifier, ItemPublisher elements now optional • PubYr attributes validate as year • PubYrFrom, PubYrToattributes added to support Journal Report 5
  • 22. COUNTER 4: Schema Changes… COUNTER Elements XML Schema: lists valid values for certain data elements • DataType: added Collection and Multimedia options • Categories: added Access_denied and removed Turnaways. • MetricType: added several new values to support reports for mobile use, the new database reports, multimedia reports and revised access denied reports.
  • 23. COUNTER 4: Procedural Changes related to SUSHI • To be COUNTER compliant, a content provider must provide a working SUSHI server • Testing the SUSHI implementation will be part of the audit
  • 24. Lessons Learned:Perspective as a Usage Consolidation Vendor COUNTER… • Offers the promise of consistency necessary for consolidated reporting • Combined with SUSHI is potentially a significant time-saver • Covers a broad range of reports providing usage for journals, books, databases
  • 25. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 26. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 27. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 28. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 29. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 30. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 31. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention
  • 32. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention • SUSHI implementations falling short
  • 33. Lessons Learned:Perspective as a Usage Consolidation Vendor SUSHI Challenges Challenges with COUNTER… • Overly complex authentication methods • • “Almost” understanding of market (most Lack of compliant reports require manual intervention SUSHI clients are hosted -- one client • SUSHI implementations falling short will harvest usage for many customers) • Support staff unfamiliar with set-up needs or operational details
  • 34. Lessons Learned:Perspective as a Usage Consolidation Vendor Challenges with COUNTER… • “Almost” compliant reports require manual intervention • SUSHI implementations falling short • Many content providers still not COUNTER compliant
  • 35. NISO SUSHI Standing Committee:Release 4 of the COUNTER Code of Practice Applying the lessons learned… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Even more resources available at the NISO SUSHI web site.
  • 36. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 37. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 38. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 39. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 40. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 41. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 42. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 43. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 44. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 45. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 46. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 47. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 48. NISO SUSHI Web Site:Support for developers and librarians Addressing the challenges… • COUNTER schema stricter • COUNTER SUSHI Implementation Profile removes ambiguity • Free web-based client available for testing or basis for development • Many resources available at the NISO SUSHI web site.
  • 49. Conclusion • SUSHI is both relevant and necessary, but we need… • more compliant content providers • with better interoperability • COUNTER Release 4 more comprehensive audit coupled with the COUNTER SUSHI Implementation Profile are keys to progress
  • 50. SUSHI: More relevant than ever Thank You! Visit the NISO SUSHI web site http://www.niso.org/workrooms/sushi Or email me at: opesch@ebsco.com
  • 51. Streamlining Stats:Managing Usage Statistics Efficiently and EffectivelyAmy Fry, Electronic Resources Coordinator, Bowling Green State University images by Ken Fager, http://www.flickr.com/photos/kenfagerdotcom/
  • 52. Usage statistics: what libraries are thinking What do I get? How do I get it? I need it now! How do I put it all together?
  • 53. Stats work today is HYBRID:part manual, part automated
  • 54. Look familiar?
  • 55. How about this?
  • 56. Just a little manual labor…
  • 57. Maybe we should just cancel everything.
  • 58. BGSU’s three principles to streamline stats Data control Organization of work Distribution of laborEverything in its (one Stats are integrated There are jobs forand only) place, all into the resource’s librarians, staff andlinked together. lifecycle. students.
  • 59. Data control ONE place for logins and passwords: our ERM.
  • 60. Data control ONE place for logins and passwords: our ERM.
  • 61. Data control ONE place for logins and passwords: our ERM.
  • 62. Data control ONE place for downloading instructions: our wikis
  • 63. dbusestats.bgsu.wikispaces.netejusestats.bgsu.wikispaces.net
  • 64. Data control ONE place for the stats files: our R drive.
  • 65. Data control ONE file for each report type
  • 66. Organization of work Stats are integrated into the lifecycle of each resource.
  • 67. Organization of work Collecting stats has been simplified • Reports are saved as downloaded (by calendar year) • Stats are collected once a year
  • 68. Distribution of labor Everyone gets to help! Librarian jobs: • Manage logins and passwords • Manage reporting • Manage the SUSHI table in the ERM • Manage stats through each resource’s lifecycle Staff jobs: • Write downloading instructions • Make changes as needed in the wiki (adding, removing, and changing resources) • Convert files XML; upload XML files into the ERM Student jobs: • Collect and save stats according to wiki instructions Other jobs: • Serials, Acquisitions & IT staff work with order records, do coverage load, and more.
  • 69. ReportingOur medium – For databases: spreadsheets, for now – For e-journals: our ERM Our philosophy – Focus on standard measures – Try to paint a picture over time – Accuracy isn’t everything
  • 70. Database usage reporting:Standard measurements
  • 71. Database usage reporting: Use over time
  • 72. Database usage reporting: Using estimates
  • 73. The “EBSCO Bump”
  • 74. E-journal usage reporting:Millennium ERM JR1 Summary Report
  • 75. E-journal reporting using the Millennium ERM– Create resource records for each journal package– Relate order records with correct from/to dates for each subscription– Download COUNTER JR1 reports in CSV format– Convert to XML using the University of Nebraska- Lincoln script (http://statsconverter4erm.unl.edu/)– Upload each XML file to the proper resource record
  • 76. Detail 1 Report for ACS Journals
  • 77. Resource-Level Statistics Export for ACS Journals
  • 78. With a little editing…
  • 79. Coming soon…• Innovative is expanding the report types that can be uploaded to its ERM in 2013!
  • 80. Feel like outsourcing?Scholarly Stats – Basic stats collection service, pay per platform – Basic collated reportsSwetsWise Selection Support – Stats collection service (pay per platform) – Options to upload data manually 360 Counter (Serials Solutions) – Unlimited platforms with annual subscription – Options to upload data manually
  • 81. Open-source ERMs• ERMes (University of Wisconsin-La Crosse) – Incorporates DB1 reports• CORAL (Notre Dame) – Incorporates JR1 reports • E-Matrix (NCSU) – Incorporates JR1 reports • CUFTS (Simon Fraser) – Incorporates JR1 reports; uses SUSHI
  • 82. Remember… You’re never, ever going to get out of collecting data altogether unless you decide to stop looking at your stats.