This document summarizes a report on the uptake of EUDAT services by the ENES climate modeling community. ENES aims to develop and evaluate climate models through model inter-comparisons. It has approximately 50 partners including universities and research organizations. The report discusses ENES' involvement with several EUDAT services, including using B2SHARE for data sharing, integrating metadata into B2FIND, and demonstrating the use of the Generic Execution Framework (GEF) for processing climate data files. The cooperation with EUDAT provides important additions to ENES' existing infrastructure by helping make data more interoperable and reusable.

1. ENES & EUDAT
Uptake Report
EUDAT Conference
24-Jan-2018
Porto
Hannes Thiemann, Heinrich Widmann (DKRZ)
Asela Rajapakse (MPI-M)

2. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 2
ENES is intended
to help in the development and evaluation of state-of-the-art climate and earth
system models
to facilitate focused model inter-comparisons in order to assess and improve these
models
ENES portal at
https://verc.enes.org/
ENES: European Network for Earth System Modelling
to encourage exchanges of software and model
results
to help in the development of high performance
computing facilities dedicated to long high-
resolution multi-model ensemble integrations.

3. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 3
Status of ENES
Approx. 50 partners
Universities
Research Organizations
Industry
In terms of IT ENES is a very mature community: HPC,
Extreme Data, Network
International research and infrastructure projects.
Earth System Grid Federation (ESGF) is the proven
and operational data infrastructure

4. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 4
ENES is one of the core communities of EUDAT
ENES partners involved
Max Planck Institute for Meteorology
Centre Européen de Recherche et de
Formation Avancée en Calcul Scientifique
(CERFACS)
Deutsches Klimarechenzentrum (DKRZ)
EUDAT & ENES

5. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 5
B2SHARE
B2HANDLE (close to production)
B2FIND (production, demo)
GEF (demo)
Areas of Involvement

6. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 6
Intended Use Case
Provide a easy to manage multi-project
B2SHARE-repository within a single B2SHARE
instance at DKRZ
Projects have different publication workflows
and different user quotas.
B2SHARE

7. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 7
Evaluation started in June 2017 with the
docker releases 2.0 and 2.0.1
Evaluation of B2SHARE

8. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 8
Multi-tenant hosting within B2SHARE not
possible.
Limited flexibility of docker version
Still demanding administration
Difficult to implement more advanced
applications using the API
B2SHARE results

9. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 9
B2HANDLE & ESGF
ESGF PID Hierarchy

10. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 10
B2HANDLE & ESGF

11. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 11
B2FIND
ENES is one of the first communities
integrated in B2FIND
~1000 ISO19139 XML records are harvested by
and published in B2FIND
Each metadata record refers via DOI to an ESM
experiment, a data collection of the order of
TB

12. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 12
Levels ofAggegation
Reuse
and
Proce
ss
data
(à
GEF ) CMIP5 Dataset
~ MB-GB
CMIP5 Dataset
~ MB-GBCMIP5 Dataset
~ MB-GB
ENES usecases
1. Publish CMIP5 Experiments in B2FIND (àB2FIND Demo)
2. Process datasets using GEF (à GEF Demo)
CMIP5 Exp
ExperimentsCMIP5 Exp
ExperimentsCMIP5 Exp
Experiments
CMIP5 Exp
ExperimentsCMIP5 Experiment
~ TB à~350 Datasets
IPCC-AR5_CMIP5
Project ~ PB
à ~ 50.000 Exp‘s
CMIP5 Dataset
~ MB-GBCMIP5 Dataset
~ MB-GB
Data Production and
Archive
Meta Data Generation
and Publishing (+ DOI)
Meta Data Ingest in
B2FIND
Open B2FIND metadataset
and access Experiment via DOI
Access Dataset and
download files
!
Find specific
CMIP5 Experiment
ISO19139 XML
MD records
Unified Common
MD schema
Facetted Search
in Discovery portal
DOI Landing page
of Experiment
Cross-domain Interoperability

13. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 13
Data Production and
Archive
Meta Data Generation
and Publishing (+ DOI)
Meta Data Ingest in
B2FIND
Open B2FIND metadataset
and access Experiment via DOI
Access Dataset and
download files
Find specific
CMIP5 Experiment
Cross-domain Interoperability
ENES goes
FAIR by using
EUDAT services !
Reuse
and
Proce
ss
data
(à
GEF )

14. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 14
The climate scientist’s perspective
14
Let‘s consider the role of an ENES climate scientist who
wishes to perform post-processing on NetCDF data
using the Climate Data Operators1 (CDOs) by
employing the EUDAT Generic Execution Framework.
1 https://code.mpimet.mpg.de/projects/cdo/

15. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 15
The input data on B2SHARE
Let‘s take a look at the NetCDF data stored in B2SHARE and
discovered with B2FIND...
15

16. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 16
GEF: architecture
To understand the demonstration, a look at the GEF architecture
is helpful:
16

17. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 17
Building a GEF service
Upload the Dockerfile and the execution script
Trigger building of the service
17

18. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 18
Building a GEF service
Let‘s take a quick look at the Dockerfile to illustrate the way it
works...
18

19. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 19
Building a GEF service...
19

20. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 20
Invoking a GEF service...
20

21. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 21
More about the GEF
21
Presentation by Xavier Pivan in the session Making
data and cloud resources interoperable using EUDAT
and EGI services held before lunch.
Presentation by Claus Zinn in the next presentation.
More information on GEF integration by communities and
GEF development are presented during the
EUDAT Conference 2018 in:

22. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 22
ENES is a very mature community with many
solutions already in place.
Cooperation with EUDAT brings important
additions to ENES.
We look forward to further collaboration with
EUDAT, also within the framework of EOSC-
hub.
Summary

23. ENES & EUDAT Uptake Report EUDAT Conference Porto 24. January 2018 2323
Thank you!