COUNTER SUSHI Oliver Pesch, ALA Annual June 2016
•
2 likes•1,305 views
presentation at ALA Annual 2016 ALCTS/LITA Electronic Resources Management Interest Group panel “Making it count: Usage statistics and electronic resources management.”
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (20)
Similar to COUNTER SUSHI Oliver Pesch, ALA Annual June 2016
Similar to COUNTER SUSHI Oliver Pesch, ALA Annual June 2016 (20)
More from National Information Standards Organization (NISO)
More from National Information Standards Organization (NISO) (20)
Recently uploaded
Recently uploaded (20)
COUNTER SUSHI Oliver Pesch, ALA Annual June 2016
- 1. “Making it count: Usage statistics and electronic resources management” COUNTER and SUSHI Oliver Pesch Chief Product Strategist EBSCO Information Services ALCTS/LITA Electronic Resources Management Interest Group
- 2. Overview • COUNTER & SUSHI review • Recent developments • What’s in the works
- 4. COUNTER... • A Code of Practice seeking “Consistent, Credible & Comparable” usage reporting usage for scholarly online information specifying... – What reports to provide – How they should be formatted – What metrics to included – Guidance for processing transaction logs – And, how they should be delivered • Audit required for compliance
- 5. “Standard” COUNTER Reports Report Name Report Description Book Report 1 Number of Successful Title Requests by Month and Title Book Report 2 Number of Successful Section Requests by Month and Title Book Report 3 Access Denied to Content Items by Month, Title and Category Book Report 4 Access Denied to Content items by Month, Platform and Category Book Report 5 Total Searches by Month and Title Database Report 1 Total Searches, Result Clicks and Record Views by Month and Database Database Report 2 Access Denied by Month, Database and Category Journal Report 1 Number of Successful Full-Text Article Requests by Month and Journal Journal Report 1 GOA Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal Journal Report 2 Access Denied to Full-Text Articles by Month, Journal and Category Journal Report 5 Number of Successful Full-Text Article Requests by Year-of-Publication (YOP) and Journal Multimedia Report 1 Number of Successful Full Multimedia Content Unit Requests by Month and Collection Platform Report 1 Total Searches, Result Clicks and Record Views by Month and Platform
- 6. “Standard” COUNTER Reports Report Name Report Description Book Report 1 Number of Successful Title Requests by Month and Title Book Report 2 Number of Successful Section Requests by Month and Title Book Report 3 Access Denied to Content Items by Month, Title and Category Book Report 4 Access Denied to Content items by Month, Platform and Category Book Report 5 Total Searches by Month and Title Database Report 1 Total Searches, Result Clicks and Record Views by Month and Database Database Report 2 Access Denied by Month, Database and Category Journal Report 1 Number of Successful Full-Text Article Requests by Month and Journal Journal Report 1 GOA Number of Successful Gold Open Access Full-Text Article Requests by Month and Journal Journal Report 2 Access Denied to Full-Text Articles by Month, Journal and Category Journal Report 5 Number of Successful Full-Text Article Requests by Year-of-Publication (YOP) and Journal Multimedia Report 1 Number of Successful Full Multimedia Content Unit Requests by Month and Collection Platform Report 1 Total Searches, Result Clicks and Record Views by Month and Platform Journal Report 1 Database Report 1 Platform Report 1 Book Reports 1 & 2
- 7. SUSHI... • A NISO/ANSI standard (Z39.93) enabling the automated exchange of COUNTER statistics via web services. • Client/server protocol – Client (ERM/usage consolidation module) requests the report via SOAP web service – Server responds with COUNTER usage in XML format – Client processes the usage • Once configured, usage harvesting can be automatic
- 8. Brief history of COUNTER and SUSHI 2016 Apr Jun Aug Oct Dec 2017 COUNTER initiative launched 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2014 2015 2016 Release 1 Journals & Databases Release 2 Journals & Databases Release 1 Books Release 3 Journals & Databases Release 4 Journals, Databases and Books SUSHI initiative launched SUSHI Draft Standard Released SUSHI Standard Released as Z39.93-2007 SUSHI Standard Revised as Z39.93-2012 SUSHI Standard Revised as Z39.93-2014 Community Website launched COUNTER SUSHI Implementation Profile Published COUNTER SUSHI Implementation Profile updated 2012 2013 SUSHI-Lite Technical Report Published DLF ERMI Report 2001 PALS International Forum on Usage
- 9. Recent Developments of Note...
- 10. Brief history of COUNTER and SUSHI 2016 Apr Jun Aug Oct Dec 2017 COUNTER initiative launched 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2014 2015 2016 Release 1 Journals & Databases Release 2 Journals & Databases Release 1 Books Release 3 Journals & Databases Release 4 Journals, Databases and Books SUSHI initiative launched SUSHI Draft Standard Released SUSHI Standard Released as Z39.93-2007 SUSHI Standard Revised as Z39.93-2012 SUSHI Standard Revised as Z39.93-2014 Community Website launched COUNTER SUSHI Implementation Profile Published COUNTER SUSHI Implementation Profile updated 2012 2013 SUSHI-Lite Technical Report Published DLF ERMI Report 2001 PALS International Forum on Usage
- 11. Release 4 of the COUNTER Code of Practice Significant features: • Sessions removed • Federated and automated searches • Result Clicks & Record Views added • SUSHI required Published 2012, became valid December 31, 2013
- 12. COUNTER SUSHI Implementation Profile • Published in 2012 to coincide with COUNTER Release 4 • Reduce ambiguity and abstraction • Provides needed guidance to SUSHI Server developers • A tool for auditors when checking for compliance
- 13. Does SUSHI really work?
- 14. Some statistics on SUSHI Harvests*... Using SUSHI successfully with more than 100 platforms! * Source: EBSCO Usage Consolidation SUSHI Harvesting Logs
- 15. Usus Community Web Site for Usage • Launched in 2015 • Supported by COUNTER; run by the community • A forum for discussions and feedback • A place to report problems and seek answers • Links to useful resources http://www.usus.org.uk/
- 16. SUSHI-Lite: lightweight version of SUSHI (2015) • Micro-service approach to exchanging COUNTER data (REST/JSON) • Much lower barrier to client development • Request snippets of usage • Embed COUNTER stats in other applications • Trial use
- 17. SUSHI-Lite Live example … https://www.jusp.mimas.ac.uk/api/sushilite/v1/Report/?Rep ort=JR1 &RequestorID=test &CustomerID=test &ItemIdentifier=journal:issn:1084-5453 &BeginDate=2012-01 &EndDate=2012-06 *based on an early prototype -- syntax and response doesn’t exactly match final recommendations
- 18. SUSHI-Lite Live example … https://www.jusp.mimas.ac.uk/api/sushilite/v1/Report/?Rep ort=JR1 &RequestorID=test &CustomerID=test &ItemIdentifier=journal:issn:1084-5453 &BeginDate=2012-01 &EndDate=2012-06 *based on an early prototype -- syntax and response doesn’t exactly match final recommendations
- 19. Renewals
- 20. Renewals
- 21. Book Report 7: Number of Successful Unique Title Requests by Month and Title in a Session • Unique book views in a session • Title only gets credited with one “request” regardless how many chapters, pages, sections viewed in that session • Replaces BR1 & BR2 • Currently optional
- 22. On the horizon
- 23. New COUNTER Web Site • Fresh look • Easier to maintain • Interactive code-of- practice • Searchable list of compliant vendors • SUSHI server registry
- 24. COUNTER Report Validator • Publically available tool • Validate SUSHI implementations • Validate COUNTER reports (XML, TSV and Excel) • Intended for: – Vendors to validate their COUNTER implementations prior to audit and release – Auditors to reduce the cost and time of the audit – Librarians to verify reports received and report errors • Being developed by COUNTER with funding provided by the EBSCO Foundation
- 25. COUNTER Release 5 Main themes & goals... • Consistency • Clarity • Simplification • Flexibility • Continuous maintenance RELEASE 5
- 26. R4: Journal Report 1 vs Database Report 1 Database Report breaks out usage by metric type. Journal (and book) reports don’t. Excel version of Journal Report 1 has totals for PDF and HTML, XML version breaks them out my month
- 27. R5: Journal Report 1 (imagined) In Release 5 all reports would include the metric type. Ideally with using the same vocabulary in Excel as XML
- 28. COUNTER Release 5 Adding clarity... • More comprehensive glossary • Describe how reports and metrics are used • Include scenarios
- 29. COUNTER Release 5 Simplification & Flexibility... • Consistency between reports • Fewer “standard” reports • Include “general” reports • Introduce filters and attributes on general reports to replace feature-specific optional reports RELEASE 5
- 30. COUNTER Release 5 Continuous maintenance... • Formalize process for enhancements • Provide mechanism for changes to be adopted without waiting for new standard RELEASE 5
- 31. References and further reading • COUNTER Code of Practice [Release 1], COUNTER, December 2002, url: http://www.projectcounter.org/codeofpractice.pdf • Electronic Resource Management: The Report of the DLF Initiative, DLF, 2004, published online at: https://old.diglib.org/pubs/dlf102/ • Press Release: NISO Initiative to Standardize Online Usage Statistics Harvesting, NISO, 2005-11-15, published online at: http://www.niso.org/news/pr/view?item_key=756120eb4a8c7871789809024f18dc81dfdfa345 • SOAP protocol definition, Wikipedia, url: https://simple.wikipedia.org/wiki/SOAP_(protocol) • eJournal Interface can Influence Usage Statistics: Implications for Libraries, Publishers, and Project COUNTER, Phil Davis, arxiv.org, July 27, 2005. url: https://arxiv.org/ftp/cs/papers/0602/0602060.pdf • Release 1 of the COUNTER Code of Practice for Books and Reference Works, COUNTER, March 2006, url: http://www.projectcounter.org/cop/books/cop_books_ref.pdf • Release 3 of the COUNTER Code of Practice for Journals and Databases, COUNTER, August 2008, url: http://www.projectcounter.org/r3/Release3D9.pdf • Hybrid Open Access Journal definition, Wikipedia, url: https://en.wikipedia.org/wiki/Hybrid_open_access_journal • COUNTER SUSHI Implementation Profile, NISO SUSHI Standing Committee, August 2012, url: http://www.niso.org/apps/group_public/download.php/13635/RP-14-2014_COUNTER_SUSHI_IP.pdf • Continuous Maintenance Procedures Standardized Usage Statistics Harvesting Initiative (SUSHI) Protocol (ANSI/NISO Z39.93), NISO website, url: http://www.niso.org/workrooms/sushi/continuous_maintenance • REST definition, Wikipedia, url: https://en.wikipedia.org/wiki/Representational_state_transfer • JSON definition, Wikipedia, url: https://en.wikipedia.org/wiki/JSON • SUSHI-Lite: Deploying SUSHI as a Lightweight Protocol for Exchanging Usage via Web Services, NISO SUSHI Standing Committee, July 2016, url: http://www.niso.org/apps/group_public/download.php/15331/NISO%20TR-06-201x%20SUSHI-Lite%20Protocol%20Draft%20For%20Trial.pdf
Editor's Notes
- Good afternoon.... I am Oliver Pesch and I am here to talk about COUNTER and SUSHI... I work as product strategist for EBSCO and I also do a lot of work with COUNTER and NISO. For COUNTER I serve on the board of directors, the Executive Committee and am chair of the COUNTER Release 5 Technical Working Group. For NISO I serve on the board of directors and am co-chair of the NISO SUSHI Standing Committee.
- Likely most of you are quite familiar with both COUNTER and SUSHI so I will keep the introduction brief so we can discuss recent developments and take a look a head to what is in store for us.
- Lets start with a quick background on COUNTER and SUSHI
- COUNTER is a Code of Practice for usage reporting – consistent, credible and compatible usage reporting. The code specifies the reports to provide, how they should be formatted, what metrics to include, how transactions should be processed and how the reports are to be delivered. Credibility is enhanced by the requirement for vendors to under go an audit.
- Here is the list of “standard” reports – those that must be supplied (when applicable) by the vendor.
- These are the ones you are probably most familiar with are... Journal Report 1, Database Report 1 and Book Reports 1 & 2 which provide the basic stats for cost-per-use analysis. And Platform Report 1 is likely the one you use when assembling overall numbers for various surveys and annual reports. We will talk about Book Report 1 & 2 a little later...
- SUSHI describes a client/server approach to harvesting COUNTER stats such that an ERM or usage consolidation application, once configured properly, can automatically request and retrieve usage stats on a regular basis (more on that later too).
- Here is a timeline that puts this in historical context... In the late 1990s, the Publishers and Libraries Services group (PALS) began looking into the challenges of usage reporting of online resources. They held an international forum on usage in London in June of 2001 which directly lead to the creation of COUNTER And the publication of the 1st code of practice in 2003. And as you can see, several releases have followed over the years For SUSHI, the 2004 DLF ERMI report was the likely its incubator... ERMs were coming on the market and the report acknowledged the promise of COUNTER as a standard for reporting usage... SUSHI came about shortly after as the automated harvesting solution to address the practicality of downloading 100s of usage reports from dozens of content providers. SUSHI became official ANSI/NISO standard in 2007.
- So with that background, lets talk about some developments of note
- From the timeline perspective - the events are COUNTER Release 4, the COUNTER SUSHI Implementation Profile, Usus web site and the SUSHI-Lite technical report...
- COUNTER Release 4 had a few impactful changes... Sessions were removed since federated searching made them an unreliable metric. Federated searches became “federated and automated” searches. Previously a discovery service or pre-selected multiple-database searches were counted as “regular” searches but counts no longer reflected user action or intent – and database search stats sky-rocketed as a result. With Release 4, “regular” searches are counted hen the user selected the database otherwise they are automated. And because “searches” were a less reliable metric, result clicks and record views were added to for database assessment reflect user actions related to a given database – a better measure of value. Plus SUSHI became a requirement for compliance... Though enforcement seemed to be a bit spotty...
- SUSHI was a frequent topic of discussion and often not in a good context... Incompatible implementations lead to a lot of frustration and while the standard itself is OK, it is abstract enough to cause confusion to implementers. The COUNTER SUSHI Implementation profile was developed by the SUSHI standing committee to provide specific guidance to developers (and auditors) to remove ambiguity.
- While there has been a sense the SUSHI isn’t working, with the efforts of various groups, things are improving...
- Lets look at some numbers... Last February, EBSCO’s SUSHI harvesting was averaging 60% FAILURE rate. Since that time we introduced an updated version of our SUSHI Client... This coupled with significant improvements on the part of publishers, we have gone from seeing a 60% FAILURE rate in February of 2015 to a 95% SUCCESS rate in February of 2016.
- In 2015 the Usus web site was launched. Supported by COUNTER but run by the community, this site provides a forum for discussion and feedback. If you have a question or problem with a SUSHI report or vendor, submit it through the feedback link and the Usus supervisory team will look into it. The track record is pretty good... But normally if a compliance issue is uncovered, the Usus team works with the vendor and the issue is usually fixed quickly.
- Also in 2015, the SUSHI Standing Committee published a technical report which described using a current-day micro-service approach to SUSHI. The goal was to lower the barrier to client development as well as offer the option to exchange snippets of usage data so one could, for example, request usage just for one journal and embed that usage in another application. SUSHI-Lite is currently in trial use.
- Here is an example of a request – it looks like a URL and it is...
- The response is in JSON format and most developers familiar with today’s web development languages can easi;y process this.
- And speaking about embedding usage... Imagine a typical renewals screen for a subscription management system... A place where renewal decisions are made...
- If we can add use and projected cost-per-use to this screen we have enhanced the user’s workflow by adding key data needed to inform the decision.
- Any of you find Book Report 1 and Book Report 2 challenging when trying to compare usage across platforms? Comparing a count of whole book downloads to chapter downloads is virtually meaningless. COUNTER has addressed this with Book Report 7 – currently an optional report that reflects unique book views within a user session. We expect this to become standard in the next release of the code of practice.
- OK... Now a quick look ahead
- COUNTER will have a new website in a couple weeks. In addition to be more “current”, it will offer interactive browsing of the code of practice and a searchable list of compliant vendors... Plus the SUSHI server registry will be integrated... And searchable by publisher.
- Also in the works... COUNTER is developing a COUNTER Report Validator. This will be an online tool that will allow vendors, auditors and librarians to perform quick validation of SUSHI implementations and regular excel COUNTER reports. The goal is to provide a consistent test environment that all implementers can use and thus reduce development time and increase compliance and interoperability.
- And then there is release 5 of the COUNTER Code of Practice. You will likely be hearing and reading more about this in the coming weeks and months. The major themes are consistency, clarity, simplification, flexibility all while introducing an continuous maintenance approach.
- Consistence... Here is Journal Report 1 and Database Report 1. Database Reports have one row per metric type, whereas, Journal Report 1 is only recording details for Full Text Requests. The Excel version of Journal Report 1 only has reporting period totals for HTML and PDF yet the XML version has these broken out by month.
- In release 5 the goal is for consistency between reports and formats. For example, all reports would include usage broken out by metric types and, where practical, the same vocabulary would be used in Excel and XML versions.
- Adding clarity through better definitions... Including intended purpose of reports and metric types... And possibly providing scenarios that visually represent a user’s trip though a site and the metrics that result for various user actions.
- While it may sound like a contradiction, we are looking to reduce the number of reports and increase flexibility. We would start by getting rid of several of the single-purpose optional reports and replace them with general reports for titles, databases, etc. General reports that can be filtered or configured to meet specific reporting goals. For example. a librarian would be able to select criteria to generate a title report with usage broken out by device type, limited to open access articles...
- Finally, the plan is for COUNTER Release 5 to include processes that allow for continuous maintenance so that, much like with SUSHI, changes can be introduced without waiting for a 5 year review of the standard.