EUDAT Porto Conference 2018

1. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
DARE: Delivering Agile Research
Excellence on European
e-Infrastructures
Vangelis Karkaletsis
vangelis@iit.demokritos.gr
National Centre for Scientific Research “Demokritos”
EUDAT Conference: Computing e-infrastructure with extreme large datasets,
Porto, 24/01/2018

2. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Motivation
• Size and complexity of scientific data
• Complexity increases exponentially with the
size and diversity of input and produced data
• Difficulty in formulating domain-specific
solutions in reproducible and reusable
ways
• Leads often to unsustainable end-user products
or long release cycles
• Big data technologies and analytics are
often not taken advantage of
• Advances in exploiting huge datasets
depend on synergy between all
categories of experts involved
Scientist
Research developer
Data engineer
Systems engineer
Data scientist

3. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Objectives
• Deliver a new working environment for
scientists and research developers
• Presenting methods in abstract terms so that domain
experts can understand, change and use them
effectively
• Providing tools that visualize the runs of these
methods in summary forms
• Improve further and integrate
• tested programmatic dataflow specification APIs, big
data technologies, provenance/data lineage solutions
• Work with two RIs:
• EPOS: European Plate Observing System
• IS-ENES: Infrastructure for the European Network of
Earth System Modelling

4. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
The Consortium
• NCSR-”Demokritos”, EL (Coordinator)
• The University of Edinburgh, UK
• INGV, IT
• CERFACS, FR
• The University of Liverpool, UK
• KNMI, NL
• GRNET S.A., EL
• Fraunhofer SCAI, DE
• “ATHENA” RIC, EL

5. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
DARE within the e-infrastructures context
• Aims to empower those who
develop methods and their
encapsulations so that they can
help their user communities.
• Delivers a unified context for
their work in order to
• support innovative data-driven
science,
• make transparent use of e-
infrastructures and other resources.

6. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Community: EPOS
• Interaction with EPOS to provide high level services
• Drive HPC simulations + data analytics
• Boundary of Data- and CPU-intensive computation
• Ensembles of simulations
• Rapid assessment of earthquake impact for emergency
coordination
• Rapid characterisation of seismic sources

7. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Community: IS-ENES
• Ensembles of climatic simulations
• Understanding and communication of uncertainties
• Meaningful federation over heterogeneous data sources
• Streamline data lifecycle
• Support Climate4Impact (C4I) services

8. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Components
• dispel4py: a high-level streaming
dataflow specification API and
library
• S-ProvFlow: combines a set of
components in support of
Reproducibility as a Service
• Exareme: A system for large-scale
dataflow processing on the cloud
• Semagrow: Semantics and linked-
data support
• BigDataEurope (BDE):
customised, cloud-ready and
modular integrator platform, for
big data management and
analytics

9. project-dare.eu Delivering Agile Research Excellence on European e-Infrastructures
Thank you!
• Questions?