Download to read offline
Uploaded byEUDAT
Data Preservation Service Area
The document discusses the EUDAT project's Data Preservation Service Area and its achievements and future plans. It summarizes that initially the service lacked metadata management, clear workflows, high-level data policies, and integration with EUDAT's Common Data Infrastructure (CDI). It is now producing features like the Data Policy Manager (DPM) for high-level data policies, improved authentication, and better documentation. Planned future work includes extending DPM's data policies, integrating a centralized PID registry, and improving metadata management, knowledge management, and interoperability. The objective is to improve DPM metadata, the PID catalog, and support FAIR data principles.
More Related Content
Similar to Data Preservation Service Area
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Data Preservation Service Area
- 1. www.eudat.euEUDAT receives fundingfrom the European Union's Horizon 2020 programme - DG CONNECT e-Infrastructures. Contract No. 654065 Data Preservation Service Area B2SAFE, Data Policy Manager, B2HANDLE, Provenance & Curation policies Claudio Cacciari (c.cacciari@cineca.it) EUDAT Conference Porto, 24th January 2018
- 2.
- 3.
- 4. Data Preservation SA- next 6 months 4 Warm-up activities (sub-task starts in M12): 1. Organise the subtask-group: DANS, CINES, STFC, GFZ. Should result in clear roles/responsibilities and PM’s for this subtask. 2. Explore the current state: RDA, CDI, B2SAFE. This should provide a clear understanding of the current infrastructure and the scope of curation and provenance. 3. Connect to WP4, WP8. This should provide a clear understanding on the types of requirements that we will be dealing with. EUDAT Kick Off - Helsinki, 26th March 2015 1. Adopt HSv8 and develop further component if required 2. Maintain EPIC APIv2 until stable alternative ready. 3. Closely follow MPA plans and initiate EUDAT policy development if there is no significant progress by the MPA at ~M6. 1. Re-design of the underlying logic of the replication mechanism 2. Optimization of the eudat rules 3. Data Policy Manager integration with GOCDB and B2ACCESS 4. Data Policy Manager improved deployment mechanism 5. HTTP API integration DCP DMP PIDs
- 5. Roadmap Data PreservationSA M13-M18 Mar 2016 – Aug 2016 M19-M24 Sep 2016 – Feb 2017 +M24 Mar 2017 - B2SAFE Core • Support for metadata ingest • Improved performance • Integration with B2ACCESS • Handle v8 support • Authorization • Local metadata store and harvesting • iRODS v4.2 support • Support for data packages (e.g. SIP and AIP) • Message bus system B2SAFE Data Policy Manager • Pre-production release DPM • Full data life cycle replication support • Integration with B2ACCESS • Integration with Data Project Coordination Portal • First production release DPM • Reviewed policy schema’s • Delegated authorization support • Initial support curation policies • Extend support for other services • Replacement of DPM agent with HTTP API • Support for hierarchical policies B2HANDLE • Support for Handle V8 • Support for EPIC v2.5 • Standardized PID profiles • PoC central PID catalog, assess requirements • Release generic Handle library (e.g pyhandle) • Release EUDAT PID library • Pre-production release central PID catalog, revisit requirements • Production release PID catalog • Pilot integration with DTR
- 6. Metadata Store Development roadmap Implementsupport of the EUDAT data model in B2SAFE and B2STAGE-HTTP Implement local MD store Make MD store harvestable Extend MD store with WUI (e.g. B2SHARE) Extend metadata support in B2SAFE and BSTAGE-HTTP Metadata Store
- 7. Summary of featuresin production: B2SAFE Update and optimization of the rules Support for metadata ingestion Integration with B2ACCESS Support to HSv8 Support to an external authorization mechanism for the rules Local metadata store and harvesting iRODS v4.2 support Support for data packages (e.g. SIP and AIP) Support to a messaging system Extend metadata store with a Web User Interface (e.g. B2SHARE) Support for B2STAGE HTTP API D D D D
- 8. Summary of featuresin production : Data Policy Manager (DPM) Integration with B2ACCESS Improved deployment mechanism Full life cycle for data replication support Integration with operational tools for the resource management Production release Reviewed policy schema Support for curation policies Extend support to other services Replacement of DPM agent with HTTP API Support for hierarchical policies D
- 9. Summary of featuresin production: Persistent Identifier Service (B2HANDLE) Adopt Handle System v8 (HSv8) EUDAT policy development Standardized PID profiles Central PID catalog Release generic Handle library Release EUDAT PID library Pilot integration with DTR D D
- 10. Lesson learned We underestimatedthe effort required by some activities The users too sometime change their mind The plan variations are also the result of a lively project with dynamic interactions among the developers and the users.
- 11. What we had Datawere replicated Persistent identifiers management was supported o No metadata management o No clear workflows o No high level data policies o No clear PID record metadata o Dependency on EPIC PID interface o Low level of integration with EUDAT CDI o Not enough documentation and training material
- 12. What we have(in production) DPM High level data policies Updated interface (native Handle System) Authentication integration Clear workflows defined Better documentation and training material EUDAT PID profile defined CDI monitoring and service catalog integration
- 13. What we have(in development) DPM Extensions to further data policies (versioning) Authentication integration PID Catalog Centralized PID registry Data publishing metadata Metadata management
- 14.
- 15. Perspectives DPM PID Catalog metadata Knowledge Management FAIR Interoperability: Interfaces: OAI-PMH, HTTPAPI Protocols Formats: METS, Data policy description Data Plans Training User documentation Data management Service management Resource management: DPMT User management: DPMT
- 16.
- 17. Thanks B2SAFE and DPM: RobertVerkerk Adil Hasan Julia Kaufhold Javier Quinteros Claudio Cacciari Pid services (B2HANDLE): Tobias Weigel Robert Verkerk Sophiane Bendouka Nicolas Liampotis Merret Buurman Data Curation and Provenance policies: Rene van Horik Linda Reijnhoudt Alexander Atamas Pascal Dugenie Javier Quinteros Vasily Bunakov
- 18.