A discovery service for UK research data 1
A discovery service for UK research data
Christopher Brown, Jisc
27/06/2017
A discovery service for UK research data 2
Content
»Background – Research @ Risk & Shared Service
»Discovery Service – pilot to service
»Demo
»Q&A
27/06/2017 A discovery service for UK research data 3
Overview: A demonstration of the UK research data discovery service,
which is providing an aggregation of research data from universities and
national subject based data centres, so that UK research data can be
discovered.
Pilot Shared Service Scope
27/06/2017 A discovery service for UK research data 5
Pilot Shared ServiceArea Other R@RWork Areas Existing Jisc Services/AgreementAreas
Credit for Architecture concepts: John Lewis (Sheffield) & Stuart Lewis (Edinburgh) http://dx.doi.org/10.6084/m9.figshare.1202230
Innovation Pipeline
27/06/2017 A discovery service for UK research data 6
All our projects are managed through our R&D pipeline. It is designed to filter the most
promising ideas and grow them to full Jisc service, and to decommission those services that
are no longer relevant. Each phase of the pipeline fulfils a specific purpose.
Digital Resources – Services
27/06/2017 A discovery service for UK research data 7
UK Research data discovery service
A platform that enables the discovery of research data
from across UK higher education institutions and data
centres
Project Page: http://jisc.ac.uk/rd/projects/uk-research-data-discovery
Blog: https://rdds.jiscinvolve.org/
Beta Service: http://researchdiscoveryservice.jisc.ac.uk
27/06/2017 A discovery service for UK research data 8
Landscape
27/06/2017 A discovery service for UK research data 9
https://researchdata.ands.org.au/
http://b2find.eudat.eu/
http://www.openresearchdata.ch/
http://etsin.avointiede.fi
http://www.europeandataportal.eu/
http://data.bnf.fr/
Benefits of research data discovery
» Increased visibility and transparency of research data helps:
› Promotion of HEI/DataCentre’s research
› Encourage re-use and sharing of data
› Validation of research
» Discovery is an important layer in research data infrastructure
» Reducing the barrier to participation in research
» Satisfying RCUK mandates and policies for open access to publicly-funded
research
» Potential increase in cross-disciplinary and cross-institutional research
» Supporting research across the research lifecycle (as part of Research @ Risk)
27/06/2017 A discovery service for UK research data 10
Pilot to Project
» Phase 1 pilot (Oct 2013 – Mar 2014):
› Digital Curation Centre (DCC) and the UK Data Archive (UKDA) pilot
› Evaluation of Australian National Data Service
› Engaged with stakeholders (HEIs and Data Centres)
› Metadata mapping and cross-walks to RIF-CS
» Phase 2 (Mar 2015 – Sept 2016):
› Jisc led with DCC and UKDA support
› Engaged with participants and gathered user stories
› Prioritised and implemented requirements
› Evaluated software and chose CKAN
› Developed Alpha system http://ckan.data.alpha.jisc.ac.uk/
› Move to Beta
27/06/2017 A discovery service for UK research data 11
Phase 2 - participating organisations
» Pilots - HEIs
› University of Hull
› University of St Andrews
› University of Glasgow
› Oxford Brookes University
› University of Edinburgh
› University of Oxford
› University of Southampton
› University of Leeds
› University of Lincoln
» Pilots – Data Centres
› Archaeology Data Centre
› Cambridge Crystallographic Data
Centre
› ISIS/ICAT - STFC
› UK Data Service
› Visual Arts Data Centre
› NERC
» Non-funded
› University of Nottingham
› University of Bath
› University of Bristol
› Lancaster University
› University of Sheffield
27/06/2017 A discovery service for UK research data 12
Metadata – a core schema
27/06/2017 A discovery service for UK research data 13
Research data discovery service
Metadata Schemas
DataCite 3 Eprints 3 /
Recollect
MODS 3.5 OAI-PMH DC Figshare UK Gemini2
(CSW)
St Andrews Glasgow Edinburgh* Lincoln Sheffield NERC (7 DCs)
Oxford Leeds Hull Oxford Brookes Cranfield
Bath Southampton Nottingham Sussex
Bristol Lancaster Stirling
Cambridge Sheffield Hallam King’s College London*
Archaeology
Data Service
Royal College of
Art
UK DataArchive
CCDC Aston Visual Arts Data Service
STFC – ISIS/ICAT Warwick
STFC – UKERC
27/06/2017 A discovery service for UK research data 14
Harvesting endpoints (http://bit.ly/RDDS3_harvest_status)
Core metadata schema (https://goo.gl/vWCX0z)
HEIs
Data Centres *Pure users
Metadata Mapping
»Core metadata schema (https://goo.gl/vWCX0z)
»Review of voting document
»UKRDDS metadata profile mapping document
(https://docs.google.com/spreadsheets/d/1mjatKZKdhp_tFm6xnYJ
FpBgPLMNDdAue9FGy-oKFBYk/edit?usp=sharing)
27/06/2017 A discovery service for UK research data 15
Gathering Requirements
» User stories via workshops
» Sector requirements reports
» Statement of Requirements
› https://drive.google.com/open?id=1lKB3rb_bmYw-
XrDJGudmNFAlUrGJ0QC0I5K4jRPtrO8
» JIRA tickets (the definitive list)
› https://jiscdev.atlassian.net/projects/RDD/issues
» Feedback via comments and workshops
» Implementing via prioritisation post-harvesting (main priority is harvesting and accuracy
of metadata)
› Review existing requirements (Must/Should/Could to be Done and review Won’t)
› Further requirements (from workshop and feedback)
› New requirements
» Will be signing off against beta system
27/06/2017 A discovery service for UK research data 16
Who’s it for? User stories
»MoSCoW prioritisation
27/06/2017 A discovery service for UK research data 17
Project / research manager
» Reporting to funders
» Find research outputs of my institution
Researcher
» Discover datasets
» Discover related objects / resources
» Find data across disciplines by location
» Find exemplar data to inspire my research
» Targeted search for topical data
» Visual search for data
» Find linked open data
» Understand metadata quality
» Understand data quality
» Show research impact
Machine
» Harvestable registry
» Show relationships between resources
Data repository
» Show repository impact
» Metadata rights respected
» Show licence and rights of data
» Index to external services
» Force refresh of registry content
System manager
» No duplicate records
» Harvest datasets
» Update platform software
Funder
» Return on investment
Phase 2 Outputs (1)
» High level evaluation of ANDS and CKAN with report
» Test instance of CKAN (alpha) with data harvested from HEIs/Data Centres
» HEI and Data Centres Requirements Reports delivered by DCC/UKDA
» User stories and use cases gathered through workshops and refined via advisory
groups
» Statement of Requirements extracted from use cases and agreed with Advisory
Groups
» JIRA – for tracking requirements, issues
» Ten Advisory Group Meetings
27/06/2017 A discovery service for UK research data 18
Phase 2 Outputs (2)
» Three workshops
» Project blog
» Scope of Datasets – Ensuring there is agreement on what datasets are harvested
» Metadata schema and mapping – Finalising the core metadata schema with
participants / advisory groups / research community
» Harvesting status and endpoints
» Alpha System – Agile development of functionality against requirements
» Final reports
27/06/2017 A discovery service for UK research data 19
Project to Service
»Phase 3 (Oct 2016 – Sept 2017):
› From test service to production ready
› Harvest from more data sources
› System testing
› Further requirements (refine and implement)
› Develop business case for service
› Deliver a more mature and tested service to Digital Resources
27/06/2017 A discovery service for UK research data 20
Phase 3 - participating organisations
»HEIs
› Sheffield Hallam
› Royal College of Art
› King’s College London
› University of Cambridge
› University of Stirling
› Aston University
› Cranfield University
› University of Sussex
»Other
› Natural History Museum
› figshare
27/06/2017 A discovery service for UK research data 21
Next steps
» Implementation of Requirements (prioritisation and development sprints)
https://jiscdev.atlassian.net/projects/RDD/
» Moved to Beta - http://researchdiscoveryservice.jisc.ac.uk/
» Resolve all harvesting issues for phase 2 participants
» Harvest all other participants
» Sprints for prioritised requirements listed in JIRA -
https://jiscdev.atlassian.net/projects/RDD/issues
» Regular releases of Beta with details sent via JISC-UKRDDS mailing list (see blog
for fortnightly updates)
» Improve usability
» Improve search functionality
27/06/2017 A discovery service for UK research data 22
Get Involved
» How you can help and participate in phase 3?
› Subscribe to JISC-UKRDDS mailing list
› Active engagement by participants or keep informed
› Check harvested metadata
› System testing and feedback
› Monitor progress, provide advice and guidance
› JISC-UKRDDS mailing list will continue as main communication outlet
› Webinars to update on progress
› Workshops as required for feedback and face-to-face discussions
27/06/2017 A discovery service for UK research data 23
Further Information
» Project page – http://jisc.ac.uk/rd/projects/uk-research-data-discovery
» Project blog – http://rdds.jiscinvolve.org/wp/
» Beta Service – http://researchdiscoveryservice.jisc.ac.uk
» Mailing list – JISC-UKRDDS@JISCMAIL.AC.UK
» JIRA – https://jiscdev.atlassian.net/projects/RDD/issues
» #jiscRDDS
» Google Drive -
https://drive.google.com/open?id=0B1NhScN5QPQ2b3k5WVRhVDlLZ28
» Padlet - http://padlet.com/chris_brown_jisc/ukrdds
» Research Data Network - https://research-data-network.readme.io/
27/06/2017 A discovery service for UK research data 24
Demo
27/06/2017 A discovery service for UK research data 25
jisc.ac.uk
Christopher Brown
Senior Co-Design Manager
christopher.brown@jisc.ac.uk
27/06/2017 A discovery service for UK research data 26

A discovery service for UK research data

  • 1.
    A discovery servicefor UK research data 1
  • 2.
    A discovery servicefor UK research data Christopher Brown, Jisc 27/06/2017 A discovery service for UK research data 2
  • 3.
    Content »Background – Research@ Risk & Shared Service »Discovery Service – pilot to service »Demo »Q&A 27/06/2017 A discovery service for UK research data 3 Overview: A demonstration of the UK research data discovery service, which is providing an aggregation of research data from universities and national subject based data centres, so that UK research data can be discovered.
  • 5.
    Pilot Shared ServiceScope 27/06/2017 A discovery service for UK research data 5 Pilot Shared ServiceArea Other R@RWork Areas Existing Jisc Services/AgreementAreas Credit for Architecture concepts: John Lewis (Sheffield) & Stuart Lewis (Edinburgh) http://dx.doi.org/10.6084/m9.figshare.1202230
  • 6.
    Innovation Pipeline 27/06/2017 Adiscovery service for UK research data 6 All our projects are managed through our R&D pipeline. It is designed to filter the most promising ideas and grow them to full Jisc service, and to decommission those services that are no longer relevant. Each phase of the pipeline fulfils a specific purpose.
  • 7.
    Digital Resources –Services 27/06/2017 A discovery service for UK research data 7
  • 8.
    UK Research datadiscovery service A platform that enables the discovery of research data from across UK higher education institutions and data centres Project Page: http://jisc.ac.uk/rd/projects/uk-research-data-discovery Blog: https://rdds.jiscinvolve.org/ Beta Service: http://researchdiscoveryservice.jisc.ac.uk 27/06/2017 A discovery service for UK research data 8
  • 9.
    Landscape 27/06/2017 A discoveryservice for UK research data 9 https://researchdata.ands.org.au/ http://b2find.eudat.eu/ http://www.openresearchdata.ch/ http://etsin.avointiede.fi http://www.europeandataportal.eu/ http://data.bnf.fr/
  • 10.
    Benefits of researchdata discovery » Increased visibility and transparency of research data helps: › Promotion of HEI/DataCentre’s research › Encourage re-use and sharing of data › Validation of research » Discovery is an important layer in research data infrastructure » Reducing the barrier to participation in research » Satisfying RCUK mandates and policies for open access to publicly-funded research » Potential increase in cross-disciplinary and cross-institutional research » Supporting research across the research lifecycle (as part of Research @ Risk) 27/06/2017 A discovery service for UK research data 10
  • 11.
    Pilot to Project »Phase 1 pilot (Oct 2013 – Mar 2014): › Digital Curation Centre (DCC) and the UK Data Archive (UKDA) pilot › Evaluation of Australian National Data Service › Engaged with stakeholders (HEIs and Data Centres) › Metadata mapping and cross-walks to RIF-CS » Phase 2 (Mar 2015 – Sept 2016): › Jisc led with DCC and UKDA support › Engaged with participants and gathered user stories › Prioritised and implemented requirements › Evaluated software and chose CKAN › Developed Alpha system http://ckan.data.alpha.jisc.ac.uk/ › Move to Beta 27/06/2017 A discovery service for UK research data 11
  • 12.
    Phase 2 -participating organisations » Pilots - HEIs › University of Hull › University of St Andrews › University of Glasgow › Oxford Brookes University › University of Edinburgh › University of Oxford › University of Southampton › University of Leeds › University of Lincoln » Pilots – Data Centres › Archaeology Data Centre › Cambridge Crystallographic Data Centre › ISIS/ICAT - STFC › UK Data Service › Visual Arts Data Centre › NERC » Non-funded › University of Nottingham › University of Bath › University of Bristol › Lancaster University › University of Sheffield 27/06/2017 A discovery service for UK research data 12
  • 13.
    Metadata – acore schema 27/06/2017 A discovery service for UK research data 13 Research data discovery service
  • 14.
    Metadata Schemas DataCite 3Eprints 3 / Recollect MODS 3.5 OAI-PMH DC Figshare UK Gemini2 (CSW) St Andrews Glasgow Edinburgh* Lincoln Sheffield NERC (7 DCs) Oxford Leeds Hull Oxford Brookes Cranfield Bath Southampton Nottingham Sussex Bristol Lancaster Stirling Cambridge Sheffield Hallam King’s College London* Archaeology Data Service Royal College of Art UK DataArchive CCDC Aston Visual Arts Data Service STFC – ISIS/ICAT Warwick STFC – UKERC 27/06/2017 A discovery service for UK research data 14 Harvesting endpoints (http://bit.ly/RDDS3_harvest_status) Core metadata schema (https://goo.gl/vWCX0z) HEIs Data Centres *Pure users
  • 15.
    Metadata Mapping »Core metadataschema (https://goo.gl/vWCX0z) »Review of voting document »UKRDDS metadata profile mapping document (https://docs.google.com/spreadsheets/d/1mjatKZKdhp_tFm6xnYJ FpBgPLMNDdAue9FGy-oKFBYk/edit?usp=sharing) 27/06/2017 A discovery service for UK research data 15
  • 16.
    Gathering Requirements » Userstories via workshops » Sector requirements reports » Statement of Requirements › https://drive.google.com/open?id=1lKB3rb_bmYw- XrDJGudmNFAlUrGJ0QC0I5K4jRPtrO8 » JIRA tickets (the definitive list) › https://jiscdev.atlassian.net/projects/RDD/issues » Feedback via comments and workshops » Implementing via prioritisation post-harvesting (main priority is harvesting and accuracy of metadata) › Review existing requirements (Must/Should/Could to be Done and review Won’t) › Further requirements (from workshop and feedback) › New requirements » Will be signing off against beta system 27/06/2017 A discovery service for UK research data 16
  • 17.
    Who’s it for?User stories »MoSCoW prioritisation 27/06/2017 A discovery service for UK research data 17 Project / research manager » Reporting to funders » Find research outputs of my institution Researcher » Discover datasets » Discover related objects / resources » Find data across disciplines by location » Find exemplar data to inspire my research » Targeted search for topical data » Visual search for data » Find linked open data » Understand metadata quality » Understand data quality » Show research impact Machine » Harvestable registry » Show relationships between resources Data repository » Show repository impact » Metadata rights respected » Show licence and rights of data » Index to external services » Force refresh of registry content System manager » No duplicate records » Harvest datasets » Update platform software Funder » Return on investment
  • 18.
    Phase 2 Outputs(1) » High level evaluation of ANDS and CKAN with report » Test instance of CKAN (alpha) with data harvested from HEIs/Data Centres » HEI and Data Centres Requirements Reports delivered by DCC/UKDA » User stories and use cases gathered through workshops and refined via advisory groups » Statement of Requirements extracted from use cases and agreed with Advisory Groups » JIRA – for tracking requirements, issues » Ten Advisory Group Meetings 27/06/2017 A discovery service for UK research data 18
  • 19.
    Phase 2 Outputs(2) » Three workshops » Project blog » Scope of Datasets – Ensuring there is agreement on what datasets are harvested » Metadata schema and mapping – Finalising the core metadata schema with participants / advisory groups / research community » Harvesting status and endpoints » Alpha System – Agile development of functionality against requirements » Final reports 27/06/2017 A discovery service for UK research data 19
  • 20.
    Project to Service »Phase3 (Oct 2016 – Sept 2017): › From test service to production ready › Harvest from more data sources › System testing › Further requirements (refine and implement) › Develop business case for service › Deliver a more mature and tested service to Digital Resources 27/06/2017 A discovery service for UK research data 20
  • 21.
    Phase 3 -participating organisations »HEIs › Sheffield Hallam › Royal College of Art › King’s College London › University of Cambridge › University of Stirling › Aston University › Cranfield University › University of Sussex »Other › Natural History Museum › figshare 27/06/2017 A discovery service for UK research data 21
  • 22.
    Next steps » Implementationof Requirements (prioritisation and development sprints) https://jiscdev.atlassian.net/projects/RDD/ » Moved to Beta - http://researchdiscoveryservice.jisc.ac.uk/ » Resolve all harvesting issues for phase 2 participants » Harvest all other participants » Sprints for prioritised requirements listed in JIRA - https://jiscdev.atlassian.net/projects/RDD/issues » Regular releases of Beta with details sent via JISC-UKRDDS mailing list (see blog for fortnightly updates) » Improve usability » Improve search functionality 27/06/2017 A discovery service for UK research data 22
  • 23.
    Get Involved » Howyou can help and participate in phase 3? › Subscribe to JISC-UKRDDS mailing list › Active engagement by participants or keep informed › Check harvested metadata › System testing and feedback › Monitor progress, provide advice and guidance › JISC-UKRDDS mailing list will continue as main communication outlet › Webinars to update on progress › Workshops as required for feedback and face-to-face discussions 27/06/2017 A discovery service for UK research data 23
  • 24.
    Further Information » Projectpage – http://jisc.ac.uk/rd/projects/uk-research-data-discovery » Project blog – http://rdds.jiscinvolve.org/wp/ » Beta Service – http://researchdiscoveryservice.jisc.ac.uk » Mailing list – JISC-UKRDDS@JISCMAIL.AC.UK » JIRA – https://jiscdev.atlassian.net/projects/RDD/issues » #jiscRDDS » Google Drive - https://drive.google.com/open?id=0B1NhScN5QPQ2b3k5WVRhVDlLZ28 » Padlet - http://padlet.com/chris_brown_jisc/ukrdds » Research Data Network - https://research-data-network.readme.io/ 27/06/2017 A discovery service for UK research data 24
  • 25.
    Demo 27/06/2017 A discoveryservice for UK research data 25
  • 26.
    jisc.ac.uk Christopher Brown Senior Co-DesignManager christopher.brown@jisc.ac.uk 27/06/2017 A discovery service for UK research data 26

Editor's Notes

  • #5 Intending to get new version of this
  • #15 KCL NERC