SlideShare a Scribd company logo
1 of 14
JUSP: how we work with your data
17 May 2022
JUSP how we work with your data - 17 May 2022
2
What we will cover
• JUSP by numbers
• How does it work? Collecting, handling and storing data
• Using your data – reports, visualisations and 3rd party products
• Working with new and existing publishers
• Working with institutions
• Where's my data?
• Q&A
3 JUSP how we work with your data - 17 May 2022
JUSP by numbers - 1
TYPES OF DATA AND REPORTS
2 COUNTER standards (Release 4, Release 5)
3 COUNTER R5 master report types (PR, TR, DR)
4 distinct types of usage – journal, book, platform,
database
7 R5 data visualisations (plus 60 for R4)
20 R5 reports – standard and custom, summary, other
CONTENT - PARTICIPANTS?
Data from 103 suppliers
Data for 350 institutions
4 JUSP how we work with your data - 17 May 2022
JUSP by numbers - 2
CONTENT – HOW MUCH?!
1,167 database tables for R5 alone
13,000 individual R5 SUSHI credentials
26,000 master reports collected monthly
5,100,000 titles stored, including 215,000 unique journal titles
730,000,000 rows of R5 data with 4 billion individual metrics
960,000,000 rows of R4 data
5 JUSP how we work with your data - 17 May 2022
How does it work? 1 – Collecting data
• JUSP works with COUNTER compliant publishers only
• Files are collected per institution, not as consortium-level data
• Data collected as .json files via SUSHI
• COUNTER 28 day rule
• Visual checks form an essential step
• Have the correct number of files downloaded, and do they "look right"?
• Issues referred to publisher or institution
• Fix credentials, other errors, re-gather files
• Move to processing phase
Check for
new data
Inspect
reports
Resolve
issues
Process
6 JUSP how we work with your data - 17 May 2022
How does it work? 2 – Processing data
• "Preflight" script checks for fundamental errors, missing or empty fields, file format issues
• What we do NOT do – check numerical values!
• Report items converted to series of identifiers and numbers for adding to JUSP
{"Platform":"Annual Reviews","Performance":[{"Period":{"Begin_Date":"2022-01-01","End_Date":"2022-01-
31"},"Instance":[{"Metric_Type":"Total_Item_Requests","Count":2},{"Metric_Type":"Total_Item_Investigations","Count":2},{"M
etric_Type":"Unique_Item_Investigations","Count":1},{"Metric_Type":"Unique_Item_Requests","Count":1}]}],"Item_ID":[{"Type
":"DOI","Value":"10.1146/anchem.816"},{"Type":"Proprietary","Value":"ar:anchem"},{"Type":"Online_ISSN","Value":"1936-
1335"},{"Type":"Print_ISSN","Value":"19361327"}],"Section_Type":"Article","Access_Method":"Regular","Access_Type":"Cont
rolled","YOP":"2009","Title":"Annual Review of Analytical
Chemistry","Publisher_ID":[{"Type":"Proprietary","Value":"ar:1015"}],"Publisher":"Annual Reviews","Data_Type":"Journal"}
title, publisher, platform, institution IDs reporting period and metric counts
305592 24 90 58 99999 6 1 2009 1 1 2022-01-01 0 0 2 2 1 1 0 0
Data, section & access types, access method, YOP
7 JUSP how we work with your data - 17 May 2022
How does it work? 3 – Storing data
• Release 5 data loaded into database tables on a per-institution and per-report type basis
• 1,000+ database tables for R5 data storage
• A typical table for a medium-sized institution with 50 publishers will contain: 5-10 million rows of data and up
to 50 million individual metrics for 2019-present
• JUSP keeps backup of the master reports collected in case of any issues
Infrastructure is hosted on Amazon Cloud and supported by other Jisc colleagues
Dummy institution (dum)
Statistics_PR_dum Statistics_TR_dum Statistics_DR_dum
8 JUSP how we work with your data - 17 May 2022
Using your data – reports, visuals, 3rd party
DB
JUSP reports – standard/custom views, summary, other
Data visualisations - Tableau
Export – CSV / TSV / email
SUSHI / 3rd party products
Internal / Jisc use
e.g. publisher negotiations
9 JUSP how we work with your data - 17 May 2022
Working with new and existing publishers
• New publishers
• Existing publishers
10 JUSP how we work with your data - 17 May 2022
Sign
agreement
Obtain
details
Test Compliance
Gather
credentials
Historic
data
Monthly
collection
Supplier
Monthly
collection
Latest data
On demand
Data
restatements
Filling gaps
New report
types
DOAJ data
Working with institutions and their data
• New institutions
• Existing institutions
11 JUSP how we work with your data - 17 May 2022
Sign
agreement
Liaison with
site
Set up
infrastructure
Gather
credentials
Collect
historic data
Monthly
collection
Institution
Monthly
collection
Latest data
On demand
Data
restatements
Filling gaps
Data / other
queries
Core title data from KB+
Where's my data?
12 JUSP how we work with your data - 17 May 2022
REPORTS
INFO
Questions
13 JUSP how we work with your data - 17 May 2022
Contact us
Email help@jisc.ac.uk
Mention JUSP in the subject line

More Related Content

Similar to JUSP: how we work with your data

Forget Big Data. It's All About Smart Data
Forget Big Data. It's All About Smart DataForget Big Data. It's All About Smart Data
Forget Big Data. It's All About Smart Data
Alan McSweeney
 
Tableau @ Spil Games
Tableau @ Spil GamesTableau @ Spil Games
Tableau @ Spil Games
Rob Winters
 
BI Portfolio
BI PortfolioBI Portfolio
BI Portfolio
tcomeaux
 

Similar to JUSP: how we work with your data (20)

Analytics Service Framework
Analytics Service Framework Analytics Service Framework
Analytics Service Framework
 
Presentasi 1 - Business Intelligence
Presentasi 1 - Business IntelligencePresentasi 1 - Business Intelligence
Presentasi 1 - Business Intelligence
 
Adding value through BI: a Jisc perspective
Adding value through BI: a Jisc perspectiveAdding value through BI: a Jisc perspective
Adding value through BI: a Jisc perspective
 
Education Seminar: Self-service BI, Logical Data Warehouse and Data Lakes
Education Seminar: Self-service BI, Logical Data Warehouse and Data LakesEducation Seminar: Self-service BI, Logical Data Warehouse and Data Lakes
Education Seminar: Self-service BI, Logical Data Warehouse and Data Lakes
 
Foundation of Business Intelligence for Business Firms .ppt
Foundation of Business Intelligence for Business Firms .pptFoundation of Business Intelligence for Business Firms .ppt
Foundation of Business Intelligence for Business Firms .ppt
 
Evaluating Transitional Agreements with Article Level Metadata
Evaluating Transitional Agreements with Article Level MetadataEvaluating Transitional Agreements with Article Level Metadata
Evaluating Transitional Agreements with Article Level Metadata
 
Forget Big Data. It's All About Smart Data
Forget Big Data. It's All About Smart DataForget Big Data. It's All About Smart Data
Forget Big Data. It's All About Smart Data
 
MSIT Adaptive Insights Brochure 2018
MSIT Adaptive Insights Brochure 2018MSIT Adaptive Insights Brochure 2018
MSIT Adaptive Insights Brochure 2018
 
Your Analytics Site Slide Deck
Your Analytics Site Slide DeckYour Analytics Site Slide Deck
Your Analytics Site Slide Deck
 
eLuminous Technologies - Business Intelligence
eLuminous Technologies - Business IntelligenceeLuminous Technologies - Business Intelligence
eLuminous Technologies - Business Intelligence
 
Big data by Mithlesh sadh
Big data by Mithlesh sadhBig data by Mithlesh sadh
Big data by Mithlesh sadh
 
Tableau @ Spil Games
Tableau @ Spil GamesTableau @ Spil Games
Tableau @ Spil Games
 
IRUS from counting clicks to COUNTER stats
IRUS from counting clicks to COUNTER statsIRUS from counting clicks to COUNTER stats
IRUS from counting clicks to COUNTER stats
 
KBC unit monitoring Petro-SIM and PI-AF
KBC unit monitoring Petro-SIM and PI-AFKBC unit monitoring Petro-SIM and PI-AF
KBC unit monitoring Petro-SIM and PI-AF
 
How to make your data count webinar, 26 Nov 2018
How to make your data count webinar, 26 Nov 2018How to make your data count webinar, 26 Nov 2018
How to make your data count webinar, 26 Nov 2018
 
BI Portfolio
BI PortfolioBI Portfolio
BI Portfolio
 
DFS22_EU Pitch Battle_David Lambermont_B.fine_041022
DFS22_EU Pitch Battle_David Lambermont_B.fine_041022DFS22_EU Pitch Battle_David Lambermont_B.fine_041022
DFS22_EU Pitch Battle_David Lambermont_B.fine_041022
 
Skillwise Big Data part 2
Skillwise Big Data part 2Skillwise Big Data part 2
Skillwise Big Data part 2
 
SharePoint Saturday - Hybrid O365 BI Solution
SharePoint Saturday - Hybrid O365 BI SolutionSharePoint Saturday - Hybrid O365 BI Solution
SharePoint Saturday - Hybrid O365 BI Solution
 
Business Intelligence Services | Business Intelligence Consulting
Business Intelligence Services | Business Intelligence ConsultingBusiness Intelligence Services | Business Intelligence Consulting
Business Intelligence Services | Business Intelligence Consulting
 

More from Jisc

More from Jisc (20)

Jisc's value to HE: the University of Sheffield
Jisc's value to HE: the University of SheffieldJisc's value to HE: the University of Sheffield
Jisc's value to HE: the University of Sheffield
 
Towards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptxTowards a code of practice for AI in AT.pptx
Towards a code of practice for AI in AT.pptx
 
Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)Jamworks pilot and AI at Jisc (20/03/2024)
Jamworks pilot and AI at Jisc (20/03/2024)
 
Wellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptxWellbeing inclusion and digital dystopias.pptx
Wellbeing inclusion and digital dystopias.pptx
 
Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)Accessible Digital Futures project (20/03/2024)
Accessible Digital Futures project (20/03/2024)
 
Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...Procuring digital preservation CAN be quick and painless with our new dynamic...
Procuring digital preservation CAN be quick and painless with our new dynamic...
 
International students’ digital experience: understanding and mitigating the ...
International students’ digital experience: understanding and mitigating the ...International students’ digital experience: understanding and mitigating the ...
International students’ digital experience: understanding and mitigating the ...
 
Digital Storytelling Community Launch!.pptx
Digital Storytelling Community Launch!.pptxDigital Storytelling Community Launch!.pptx
Digital Storytelling Community Launch!.pptx
 
Open Access book publishing understanding your options (1).pptx
Open Access book publishing understanding your options (1).pptxOpen Access book publishing understanding your options (1).pptx
Open Access book publishing understanding your options (1).pptx
 
Scottish Universities Press supporting authors with requirements for open acc...
Scottish Universities Press supporting authors with requirements for open acc...Scottish Universities Press supporting authors with requirements for open acc...
Scottish Universities Press supporting authors with requirements for open acc...
 
How Bloomsbury is supporting authors with UKRI long-form open access requirem...
How Bloomsbury is supporting authors with UKRI long-form open access requirem...How Bloomsbury is supporting authors with UKRI long-form open access requirem...
How Bloomsbury is supporting authors with UKRI long-form open access requirem...
 
Jisc Northern Ireland Strategy Forum 2023
Jisc Northern Ireland Strategy Forum 2023Jisc Northern Ireland Strategy Forum 2023
Jisc Northern Ireland Strategy Forum 2023
 
Jisc Scotland Strategy Forum 2023
Jisc Scotland Strategy Forum 2023Jisc Scotland Strategy Forum 2023
Jisc Scotland Strategy Forum 2023
 
Jisc stakeholder strategic update 2023
Jisc stakeholder strategic update 2023Jisc stakeholder strategic update 2023
Jisc stakeholder strategic update 2023
 
JISC Presentation.pptx
JISC Presentation.pptxJISC Presentation.pptx
JISC Presentation.pptx
 
Community-led Open Access Publishing webinar.pptx
Community-led Open Access Publishing webinar.pptxCommunity-led Open Access Publishing webinar.pptx
Community-led Open Access Publishing webinar.pptx
 
The Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptxThe Open Access Community Framework (OACF) 2023 (1).pptx
The Open Access Community Framework (OACF) 2023 (1).pptx
 
Are we onboard yet University of Sussex.pptx
Are we onboard yet University of Sussex.pptxAre we onboard yet University of Sussex.pptx
Are we onboard yet University of Sussex.pptx
 
JiscOAWeek_LAIR_slides_October2023.pptx
JiscOAWeek_LAIR_slides_October2023.pptxJiscOAWeek_LAIR_slides_October2023.pptx
JiscOAWeek_LAIR_slides_October2023.pptx
 
UWP OA Week Presentation (1).pptx
UWP OA Week Presentation (1).pptxUWP OA Week Presentation (1).pptx
UWP OA Week Presentation (1).pptx
 

Recently uploaded

會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
中 央社
 
ppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyesppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyes
ashishpaul799
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
heathfieldcps1
 

Recently uploaded (20)

會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽會考英聽
 
Discover the Dark Web .pdf InfosecTrain
Discover the Dark Web .pdf  InfosecTrainDiscover the Dark Web .pdf  InfosecTrain
Discover the Dark Web .pdf InfosecTrain
 
Word Stress rules esl .pptx
Word Stress rules esl               .pptxWord Stress rules esl               .pptx
Word Stress rules esl .pptx
 
factors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptxfactors influencing drug absorption-final-2.pptx
factors influencing drug absorption-final-2.pptx
 
Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).Dementia (Alzheimer & vasular dementia).
Dementia (Alzheimer & vasular dementia).
 
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdfPost Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
 
....................Muslim-Law notes.pdf
....................Muslim-Law notes.pdf....................Muslim-Law notes.pdf
....................Muslim-Law notes.pdf
 
Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17Features of Video Calls in the Discuss Module in Odoo 17
Features of Video Calls in the Discuss Module in Odoo 17
 
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
 Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
Envelope of Discrepancy in Orthodontics: Enhancing Precision in Treatment
 
An overview of the various scriptures in Hinduism
An overview of the various scriptures in HinduismAn overview of the various scriptures in Hinduism
An overview of the various scriptures in Hinduism
 
How to Analyse Profit of a Sales Order in Odoo 17
How to Analyse Profit of a Sales Order in Odoo 17How to Analyse Profit of a Sales Order in Odoo 17
How to Analyse Profit of a Sales Order in Odoo 17
 
“O BEIJO” EM ARTE .
“O BEIJO” EM ARTE                       .“O BEIJO” EM ARTE                       .
“O BEIJO” EM ARTE .
 
How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17
 
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptxslides CapTechTalks Webinar May 2024 Alexander Perry.pptx
slides CapTechTalks Webinar May 2024 Alexander Perry.pptx
 
ppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyesppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyes
 
The Last Leaf, a short story by O. Henry
The Last Leaf, a short story by O. HenryThe Last Leaf, a short story by O. Henry
The Last Leaf, a short story by O. Henry
 
BỘ LUYỆN NGHE TIẾNG ANH 8 GLOBAL SUCCESS CẢ NĂM (GỒM 12 UNITS, MỖI UNIT GỒM 3...
BỘ LUYỆN NGHE TIẾNG ANH 8 GLOBAL SUCCESS CẢ NĂM (GỒM 12 UNITS, MỖI UNIT GỒM 3...BỘ LUYỆN NGHE TIẾNG ANH 8 GLOBAL SUCCESS CẢ NĂM (GỒM 12 UNITS, MỖI UNIT GỒM 3...
BỘ LUYỆN NGHE TIẾNG ANH 8 GLOBAL SUCCESS CẢ NĂM (GỒM 12 UNITS, MỖI UNIT GỒM 3...
 
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...Navigating the Misinformation Minefield: The Role of Higher Education in the ...
Navigating the Misinformation Minefield: The Role of Higher Education in the ...
 
The basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptxThe basics of sentences session 4pptx.pptx
The basics of sentences session 4pptx.pptx
 
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdfPost Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
Post Exam Fun(da) Intra UEM General Quiz 2024 - Prelims q&a.pdf
 

JUSP: how we work with your data

  • 1. JUSP: how we work with your data 17 May 2022
  • 2. JUSP how we work with your data - 17 May 2022 2
  • 3. What we will cover • JUSP by numbers • How does it work? Collecting, handling and storing data • Using your data – reports, visualisations and 3rd party products • Working with new and existing publishers • Working with institutions • Where's my data? • Q&A 3 JUSP how we work with your data - 17 May 2022
  • 4. JUSP by numbers - 1 TYPES OF DATA AND REPORTS 2 COUNTER standards (Release 4, Release 5) 3 COUNTER R5 master report types (PR, TR, DR) 4 distinct types of usage – journal, book, platform, database 7 R5 data visualisations (plus 60 for R4) 20 R5 reports – standard and custom, summary, other CONTENT - PARTICIPANTS? Data from 103 suppliers Data for 350 institutions 4 JUSP how we work with your data - 17 May 2022
  • 5. JUSP by numbers - 2 CONTENT – HOW MUCH?! 1,167 database tables for R5 alone 13,000 individual R5 SUSHI credentials 26,000 master reports collected monthly 5,100,000 titles stored, including 215,000 unique journal titles 730,000,000 rows of R5 data with 4 billion individual metrics 960,000,000 rows of R4 data 5 JUSP how we work with your data - 17 May 2022
  • 6. How does it work? 1 – Collecting data • JUSP works with COUNTER compliant publishers only • Files are collected per institution, not as consortium-level data • Data collected as .json files via SUSHI • COUNTER 28 day rule • Visual checks form an essential step • Have the correct number of files downloaded, and do they "look right"? • Issues referred to publisher or institution • Fix credentials, other errors, re-gather files • Move to processing phase Check for new data Inspect reports Resolve issues Process 6 JUSP how we work with your data - 17 May 2022
  • 7. How does it work? 2 – Processing data • "Preflight" script checks for fundamental errors, missing or empty fields, file format issues • What we do NOT do – check numerical values! • Report items converted to series of identifiers and numbers for adding to JUSP {"Platform":"Annual Reviews","Performance":[{"Period":{"Begin_Date":"2022-01-01","End_Date":"2022-01- 31"},"Instance":[{"Metric_Type":"Total_Item_Requests","Count":2},{"Metric_Type":"Total_Item_Investigations","Count":2},{"M etric_Type":"Unique_Item_Investigations","Count":1},{"Metric_Type":"Unique_Item_Requests","Count":1}]}],"Item_ID":[{"Type ":"DOI","Value":"10.1146/anchem.816"},{"Type":"Proprietary","Value":"ar:anchem"},{"Type":"Online_ISSN","Value":"1936- 1335"},{"Type":"Print_ISSN","Value":"19361327"}],"Section_Type":"Article","Access_Method":"Regular","Access_Type":"Cont rolled","YOP":"2009","Title":"Annual Review of Analytical Chemistry","Publisher_ID":[{"Type":"Proprietary","Value":"ar:1015"}],"Publisher":"Annual Reviews","Data_Type":"Journal"} title, publisher, platform, institution IDs reporting period and metric counts 305592 24 90 58 99999 6 1 2009 1 1 2022-01-01 0 0 2 2 1 1 0 0 Data, section & access types, access method, YOP 7 JUSP how we work with your data - 17 May 2022
  • 8. How does it work? 3 – Storing data • Release 5 data loaded into database tables on a per-institution and per-report type basis • 1,000+ database tables for R5 data storage • A typical table for a medium-sized institution with 50 publishers will contain: 5-10 million rows of data and up to 50 million individual metrics for 2019-present • JUSP keeps backup of the master reports collected in case of any issues Infrastructure is hosted on Amazon Cloud and supported by other Jisc colleagues Dummy institution (dum) Statistics_PR_dum Statistics_TR_dum Statistics_DR_dum 8 JUSP how we work with your data - 17 May 2022
  • 9. Using your data – reports, visuals, 3rd party DB JUSP reports – standard/custom views, summary, other Data visualisations - Tableau Export – CSV / TSV / email SUSHI / 3rd party products Internal / Jisc use e.g. publisher negotiations 9 JUSP how we work with your data - 17 May 2022
  • 10. Working with new and existing publishers • New publishers • Existing publishers 10 JUSP how we work with your data - 17 May 2022 Sign agreement Obtain details Test Compliance Gather credentials Historic data Monthly collection Supplier Monthly collection Latest data On demand Data restatements Filling gaps New report types DOAJ data
  • 11. Working with institutions and their data • New institutions • Existing institutions 11 JUSP how we work with your data - 17 May 2022 Sign agreement Liaison with site Set up infrastructure Gather credentials Collect historic data Monthly collection Institution Monthly collection Latest data On demand Data restatements Filling gaps Data / other queries Core title data from KB+
  • 12. Where's my data? 12 JUSP how we work with your data - 17 May 2022 REPORTS INFO
  • 13. Questions 13 JUSP how we work with your data - 17 May 2022
  • 14. Contact us Email help@jisc.ac.uk Mention JUSP in the subject line