Jo Lambert Jisc Paul Needham University of Cranfield The success of COUNTER in supporting adoption of a standard to measure e-resource usage over the past 15 years is apparent. The prevalence of global OA policies and mandates, and the role of institutional repositories within this context prompts demand for more granular metrics. It also raises the profile of data sharing of item level usage and research data metrics. The need for reliable and authoritative measures is key. This burgeoning interest is complemented by a number of initiatives to explore the measurement and tracking of usage of a broad range of objects outside traditional publisher platforms. Drawing on examples such as OpenAIRE, IRUSdata-UK, Crossref’s distributed usage logging and DOI event tracker projects, COAR Next Generation Repositories and IRUS-UK, this session will provide an update on progress in this area, discuss some challenges and current approaches to tackling them

1. UKSG Annual Conference: 8 & 10 April 2019
Institutional repositories, item and research data metrics
By dirkcuys , CC BY-SA 2.0, https://flic.kr/p/jAZBNr/
Paul Needham & Jo Lambert

2. Session summary
•Context
•Standards and protocols
•Tools and initiatives
•Your questions
2

3. Context

4. Context
•Transition to OA: institutional, researcher and funder requirements
•Usage statistics to demonstrate value and impact
•How do you gain a complete picture of usage when availability of
content is disparate and fragmented?
4

5. Standards and protocols

6. Standards and protocols
6
•COUNTER
•COUNTER Code of Practice release 5
•COUNTER Code of Practice for Research Data Usage Metrics release 1
•SUSHI
•Tracker protocol
•DUL protocol

7. Standards and protocols
7
COUNTER
•COUNTER provides a usage statistics infrastructure to support publishers,
aggregators, libraries and third parties
•The de facto global standard for counting usage of scholarly resources
•First Code of Practice was published in 2003
•A number of iterations since then in response to a changing environment and
evolving requirements
•The latest CoP is the COUNTER Code of Practice release 5

8. Standards and protocols
8
COUNTER Code of Practice release 5
•Builds on previous codes of practice
•Standardises usage metrics for e-resources, including:
journals, books, databases, platforms, multimedia and articles
•Focuses on improving the clarity, consistency, and comparability of usage
reporting
•4 Master Reports, StandardViews for common use cases, allows for Custom
Reports andViews
•Incorporates reporting for Institutional Repositories and Open Access publishers

9. Standards and protocols
9
COUNTER Code of Practice for Research Data Usage Metrics release 1
•New, experimental code of practice
•First attempt to support consistent reporting of research data usage
•Developed in a collaboration between COUNTER, Make DataCount (California
Digital Library, DataCite, and DataONE) and Jisc/IRUS
•Aligned with COUNTER CoP release 5 – as much as possible
•But . . . research data does involve certain unique aspects that are handled
specifically through this Code of Practice

10. Standards and protocols
10
SUSHI Protocol
•Standardized Usage Harvesting Initiative
•Developed by NISO and COUNTER
•Automated machine to machine transmission of usage reports
•Key to handling large amounts of usage data
•Up to COUNTER R4 the protocol was based on SOAP and XML
•Now for R5, it’s been transformed into a RESTful API returning JSON

11. Standards and protocols
11
Tracker Protocol
•Developed by COUNTER and other partners during the PIRUS and PIRUS2
projects
•Key data about a download from a repository is bundled into an OpenURL query
string
•What was downloaded, when, from where, etc.
•Sent to a third-party server for processing into COUNTER statistics
•Used to demonstrate technical ability to create, record and consolidate usage
statistics for individual articles hosted both by repositories and publishers

12. Standards and protocols
12
Tracker Protocol
•Subsequently included in the COUNTER Code of Practice for Articles
•Used by the IRUS family of services to get raw download data from repositories
in the UK and beyond
•https://irus.jisc.ac.uk/counter/?url_ver=Z39.88-2004&req_id=111.222.333.444&req_dat=Mozilla/5.0
(iPhone; CPU iPhone OS 12_1_4 like Mac OS
X)&rft.artnum=oai:www.repo.ac.uk:10399/2321&rfr_dat=&rfr_id=www.repo.ac.uk&url_tim=2019-
04-04T09:34:35Z&svc_dat=https://www.repo.ac.uk/example.pdf

13. Standards and protocols
13
DUL Protocol
•A DUL Message is defined as a JOSE (Javascript Object Signing and Encryption)
object
•message is a self-contained package
•contains all of the context required to process and interpret it
•As a JOSE object, the DUL Message has three sections:
•the header, which indicates the authenticity of the message
•the payload, the content of the message: a DUL Envelope
•the signature, which establishes the authenticity of the message and its
sender.

14. Standards and protocols
14
{
"Transaction_ID": "12345",
"Transaction_DateTime": "2017-06-06T11:28:57Z",
"Transaction_Type": "Request",
"Transaction_Access_Type": "Controlled",
"Transaction_Access_Method": "Regular",
"Referring_URL": "url user clicked to generate transaction",
"User_Agent": "...",
"Session_ID": "sessionID or surrogate derived from
useragent+IP”,
"Encoded_IP": "non-reversible encryption of IP",
"IP_ClassC": "10.20.30",
"Item_Platform": "sender-platform-name",
"Item_ID": {
"type": "DOI",
"value": "123.232/123213.22"
},
"Article_Version": "ALPSP/NISO code indicating the version
of the work. (e.g.VoR)",
"Org_ID": {
"type": "ISNI",
"value": "123-123-14-123"
}
}
DUL Protocol

15. Tools and initiatives

16. Tools and initiatives: IRUS
16
IRUS family of services
IRUS
IRUS-UK
IRUSdata-UK (beta)
IRUS-UvA
IRUS-OAPEN
IRUS-CORE
IRUS-ANZ (pilot)
IRUS-USA (pilot)

17. Tools and initiatives: IRUS-UK
17
•A national aggregation service enabling
IRs to share/expose usage statistics at
the Individual item level, based on a
global standard – COUNTER
•Collect raw download data from IRs for
all item types within repositories
•Process those raw data into COUNTER-
conformant statistics

18. Tools and initiatives: IRUS-UK
18
Gathering data
•Simple method to gather raw download data: theTracker Protocol
•On download, message sent to IRUS-UK server with some basic details about
that download
•Accomplished by adding a small piece of code to repository software
•You can find out more about our implementation at
• https://irus.jisc.ac.uk/support/toolbox/

19. Tools and initiatives: IRUS-UK
19
Processing data
•Release 4 of the COUNTER Code of Practice for e-Resources
•Release 1 of the COUNTER Code of Practice for Articles
•We remove robot entries using the COUNTER user agent exclusion list
(https://github.com/atmire/COUNTER-Robots)
•But we need to do more in the Open Access world
•Additional exclusions – overactive IPs, fake referrers, ‘bad’ bots
This year
•Developing IRUS-UK to follow the COUNTER Code of Practice release 5

20. Tools and initiatives: IRUS-UK
20
Data access and presentation
•Web portal
•Tabular reports
•Consolidation of repository and CORE
statistics
•ORCiD
•Visualisations
•Altmetric & Dimensions badges
•SUSHI Lite API
•Widget

21. Tools and initiatives: IRUSdata-UK
21
•A beta sister service to IRUS-UK
•Intended specifically for repositories that host research data
•Uses same tracker protocol as IRUS-UK but with different endpoint
•Processing is similar to IRUS-UK
•But reports down to individual file level rather than just reporting at item level
•Pioneering effort, test bed for research data statistics
•Directly led to work involving Jisc, Making DataCount (MDC) and COUNTER

22. Tools and initiatives: IRUSdata-UK
22
Coming this year
•Developing IRUSdata-UK to follow the COUNTER Code of Practice for
Research Data
•Plan to include IRUSdata as a component of Jisc’s open data research hub

23. Tools and initiatives: OpenAIRE
23
•An initiative supported by the European Commission
•General remit to implement EC Open Access and Open Data policies and
mandates
•The OpenAIRE Usage Statistics Service
•Contributes towards impact evaluation of usage activity in Open Access
Repositories
•Gathers usage data and consolidates usage statistics from a distributed network
of data providers
•UK via IRUS-UK
•Europe
•South America via La Referencia

24. Tools and initiatives: OpenAIRE
24
•Jisc,OpenAIRE, COAR collaborate and cooperate
•Like IRUS, use open standards and protocols such as
•COUNTER
•Tracker protocol, though a different implementation
•Usage statistics available in the OpenAIRE Content Provider Dashboard.
•Common goal – a global central hub for Open Access Usage Statistics
•OpenAIRE Advance Next Generation Repositories task
•Signposting, ResourceSync

25. Tools and initiatives: COAR Next Generation Repositories
25
Confederation of Open Access Repositories working group
•Define priority functionalities for repositories in the future – ‘behaviours’:
•Exposing Identifiers; Declaring Licenses at the Resource Level; DiscoveryThrough Navigation;
Interacting with Resources (Annotation, Commentary, and Review); ResourceTransfer; Batch
Discovery; Collecting and Exposing Activities; Identification of Users; Authentication of Users;
Exposing Standardized Usage Metrics; Preserving Resources
•Will facilitate the development of new services on top of the collective
network, including social networking, peer review, notifications, and usage
assessment
•Group has examined available technologies, standards and protocols to
support each behaviour. See http://ngr.coar-repositories.org/

26. Tools and initiatives: Crossref DUL
26
Distributed Usage Logging initiative
•Collaboration between COUNTER,Crossref members, and scholarly technology
& service providers
•Researchers use “alternative” (non-publisher) platforms to store, access and
share literature
•Repositories, networking sites (ResearchGate, Mendeley), etc.
•DUL allows
•publishers to capture traditional usage activity related to their content,
happening on sites other than their own
•reporting of “total usage” regardless of where that usage happens

27. Tools and initiatives: Crossref DUL
27
Distributed Usage Logging initiative
•DUL working group
•Provide a private peer-to-peer channel for the secure exchange and processing
of COUNTER-compliant private usage records from hosting platforms to
publishers
•Utilises the DUL protocol
•There are already proof-of-concept implementations
•Crossref Board will be discussing its long-term future this year

28. Tools and initiatives: DOI event tracker
28
•Crossref initiative started about 4 years ago
•There’s interest in better understanding how scholarly research is used
•Scholarly content
•Discussed by scholars outside the formal literature
•By others beyond the academic community
•Aim to track activity surrounding a research work from potentially any web
source where an event is associated with a DOI
•Social media: Facebook,Twitter,Tumblr, Reddit; blogs . . .

29. Tools and initiatives: Crossref Event Data service
29
•DET has become the Crossref Event Data service
•Has many data sources including:Crossref, DataCite, Reddit,Twitter . . .
•For anyone who wants to
•build tools to enable their organization to see ‘a fuller picture’
•or focus in on any specific pixel of that picture
• Publishers, Service Providers, Bibliometricians, Funders, anyone . . .
• Find out more at:
•https://www.crossref.org/services/event-data/

30. Summary
30
•Collaborate, engage, share
•Adopt and drive development of standards
•Build tools that exploit those standards and demonstrate examples of
effective use

31. Links
31
COUNTER: https://www.projectcounter.org
IRUS-UK: https://irus.jisc.ac.uk
OpenAIRE usage statistics guidelines: https://openaire.github.io/usage-
statistics-guidelines/
COAR Next Generation Repositories: http://ngr.coar-repositories.org/
Crossref Event Data service: https://www.crossref.org/services/event-data/

32. Your questions