JUSP: how we work with your data
17 May 2022

JUSP how we work with your data - 17 May 2022
2

What we will cover
• JUSP by numbers
• How does it work? Collecting, handling and storing data
• Using your data – reports, visualisations and 3rd party products
• Working with new and existing publishers
• Working with institutions
• Where's my data?
• Q&A
3 JUSP how we work with your data - 17 May 2022

JUSP by numbers - 1
TYPES OF DATA AND REPORTS
2 COUNTER standards (Release 4, Release 5)
3 COUNTER R5 master report types (PR, TR, DR)
4 distinct types of usage – journal, book, platform,
database
7 R5 data visualisations (plus 60 for R4)
20 R5 reports – standard and custom, summary, other
CONTENT - PARTICIPANTS?
Data from 103 suppliers
Data for 350 institutions
4 JUSP how we work with your data - 17 May 2022

JUSP by numbers - 2
CONTENT – HOW MUCH?!
1,167 database tables for R5 alone
13,000 individual R5 SUSHI credentials
26,000 master reports collected monthly
5,100,000 titles stored, including 215,000 unique journal titles
730,000,000 rows of R5 data with 4 billion individual metrics
960,000,000 rows of R4 data
5 JUSP how we work with your data - 17 May 2022

How does it work? 1 – Collecting data
• JUSP works with COUNTER compliant publishers only
• Files are collected per institution, not as consortium-level data
• Data collected as .json files via SUSHI
• COUNTER 28 day rule
• Visual checks form an essential step
• Have the correct number of files downloaded, and do they "look right"?
• Issues referred to publisher or institution
• Fix credentials, other errors, re-gather files
• Move to processing phase
Check for
new data
Inspect
reports
Resolve
issues
Process
6 JUSP how we work with your data - 17 May 2022

How does it work? 2 – Processing data
• "Preflight" script checks for fundamental errors, missing or empty fields, file format issues
• What we do NOT do – check numerical values!
• Report items converted to series of identifiers and numbers for adding to JUSP
{"Platform":"Annual Reviews","Performance":[{"Period":{"Begin_Date":"2022-01-01","End_Date":"2022-01-
31"},"Instance":[{"Metric_Type":"Total_Item_Requests","Count":2},{"Metric_Type":"Total_Item_Investigations","Count":2},{"M
etric_Type":"Unique_Item_Investigations","Count":1},{"Metric_Type":"Unique_Item_Requests","Count":1}]}],"Item_ID":[{"Type
":"DOI","Value":"10.1146/anchem.816"},{"Type":"Proprietary","Value":"ar:anchem"},{"Type":"Online_ISSN","Value":"1936-
1335"},{"Type":"Print_ISSN","Value":"19361327"}],"Section_Type":"Article","Access_Method":"Regular","Access_Type":"Cont
rolled","YOP":"2009","Title":"Annual Review of Analytical
Chemistry","Publisher_ID":[{"Type":"Proprietary","Value":"ar:1015"}],"Publisher":"Annual Reviews","Data_Type":"Journal"}
title, publisher, platform, institution IDs reporting period and metric counts
305592 24 90 58 99999 6 1 2009 1 1 2022-01-01 0 0 2 2 1 1 0 0
Data, section & access types, access method, YOP
7 JUSP how we work with your data - 17 May 2022

How does it work? 3 – Storing data
• Release 5 data loaded into database tables on a per-institution and per-report type basis
• 1,000+ database tables for R5 data storage
• A typical table for a medium-sized institution with 50 publishers will contain: 5-10 million rows of data and up
to 50 million individual metrics for 2019-present
• JUSP keeps backup of the master reports collected in case of any issues
Infrastructure is hosted on Amazon Cloud and supported by other Jisc colleagues
Dummy institution (dum)
Statistics_PR_dum Statistics_TR_dum Statistics_DR_dum
8 JUSP how we work with your data - 17 May 2022

Using your data – reports, visuals, 3rd party
DB
JUSP reports – standard/custom views, summary, other
Data visualisations - Tableau
Export – CSV / TSV / email
SUSHI / 3rd party products
Internal / Jisc use
e.g. publisher negotiations
9 JUSP how we work with your data - 17 May 2022

Working with new and existing publishers
• New publishers
• Existing publishers
10 JUSP how we work with your data - 17 May 2022
Sign
agreement
Obtain
details
Test Compliance
Gather
credentials
Historic
data
Monthly
collection
Supplier
Monthly
collection
Latest data
On demand
Data
restatements
Filling gaps
New report
types
DOAJ data

Working with institutions and their data
• New institutions
• Existing institutions
11 JUSP how we work with your data - 17 May 2022
Sign
agreement
Liaison with
site
Set up
infrastructure
Gather
credentials
Collect
historic data
Monthly
collection
Institution
Monthly
collection
Latest data
On demand
Data
restatements
Filling gaps
Data / other
queries
Core title data from KB+

Where's my data?
12 JUSP how we work with your data - 17 May 2022
REPORTS
INFO

Questions
13 JUSP how we work with your data - 17 May 2022

Contact us
Email help@jisc.ac.uk
Mention JUSP in the subject line