This document discusses Work Package 5 (WP5) of the ESCAPE project, which aims to develop a Science Analysis Platform (ESAP) to facilitate access to data and software for researchers. WP5 involves several tasks: T5.1 focuses on data aggregation and staging; T5.2 on software deployment and virtualization; T5.3 on an analysis interface and workflows; and T5.4 on integrating the platform with high-performance computing infrastructures. The document outlines the goals, components, milestones, and timeline of WP5 and its coordination with other Work Packages to develop the ESAP.
1. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
ESCAPE - The European Science Cluster of Astronomy & Particle Physics ESFRI Research Infrastructures has received funding from
the European Union’s Horizon 2020 research and innovation programme under the Grant Agreement n° 824064.
WP5 - ESAP
ESFRI Science Analysis Platform
Zheng Meyer-Zhao
ASTRON, The Netherlands
ESCAPE Kick-off Meeting Annecy, 8 February 2019
2. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
28 February 2019 meyer@astron.nl8 February 2019
Integration with Data Lake - distributed computing and storage
Access to
software &
services in
ESCAPE-EOSC
catalogue
Connect science
platform with
existing
astronomical data
archives and VO-
enabled data
collections
3. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
8 February 2019 meyer@astron.nl 3
T5.1 - Data aggregation and
staging
● Stage data in the Data Lake (WP2)
● Data discovery, VO (WP4) - to be expanded
● Dynamically allocate user workspace across distributed
infrastructure
● Tools to estimate availability & latency
● Demonstrate for a range of data collections (CTA, ESO,
EST, FAIR, JIVE, LOFAR,...)
4. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
8 February 2019 meyer@astron.nl 4
T5.2 - Software deployment and
virtualisation
● Integrate software and service repository (WP3),
allow access to software components developed by
ESFRIs
● Provide access to software repository metadata
● Support containerisation of additional tools
● Demonstrate with variety of examples (ESO, FAIR,
JIVE, LOFAR)
5. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
8 February 2019 meyer@astron.nl 5
T5.3 – Analysis interface, work flows and
reproducibility
● Interactive analysis interface which Integrates data
access & staging (T5.1)
● Provides access to EOSC software repository (T5.2)
● Simplify porting workflows to science platform
environment
● support common deployment language (e.g. CWL)
● deploy across EOSC infrastructure
● promote preservation & sharing of workflows
● Start with small number of representative workflows
● Evaluate performance, monitor compliance w/ FAIR
principles
6. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
8 February 2019 meyer@astron.nl 6
T5.4 - Integration with HPC and HTC
infrastructures
● Deploy user-initiated workflows on HPC and HTC
infrastructure
● but... maintain interactivity and responsiveness
● Obviously close links with WP2 – integrate Science
Platform with Data Lake
● Expand number of ESFRIs supported
7. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
78 February 2019 meyer@astron.nl
Links with other ESCAPE WPs
• WP1 - General EOSC policy for services &
infrastructure access
• WP2 - Integration with Data Lake - distributed
computing and storage, AAI
• WP3 - Access to software & services in ESCAPE-EOSC
catalogue
• WP4 - connect science platform with existing
astronomical data archives and VO-enabled data
collections
8 February 2019
8. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
88 February 2019 meyer@astron.nl
ESAP Components
8 February 2019
9. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
98 February 2019 meyer@astron.nl
ESAP Tasks
8 February 2019
10. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
108 February 2019 meyer@astron.nl
Related tools/projects
8 February 2019
11. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
118 February 2019 meyer@astron.nl
ESAP and other Work Packages
8 February 2019
12. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
128 February 2019 meyer@astron.nl
Deliverables
8 February 2019
13. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
138 February 2019 meyer@astron.nl
Milestones
8 February 2019
14. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
148 February 2019 meyer@astron.nl
(ESFRI) Use Cases
8 February 2019
● Identify one or more use case(s) that represent:
● Your research/instrument data
● Data processing requirements
● Data distribution requirements
● ...
● We will provide a template for preparing the use cases
● Provided use cases will be used as input for the ESAP
design
15. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
158 February 2019 meyer@astron.nl
TODO’s
8 February 2019
● Identify WP5 communication channel
● Identify task leader from Task leading institutes
● Task 5.1: (CERN, CNRS-LAPP, FAIR GMBH, NWO-I-Nikhef)
● Task 5.2: (FAIR GMBH, JIVE, NWO-I-ASTRON)
● Task 5.3: Zheng Meyer-Zhao (NWO-I-ASTRON)
● Task 5.4: Giuliano Taffoni (INAF)
● Task 5.5: Michiel van Haarlem (NWO-I-ASTRON)
● Within each task:
● Coordination of regular Telecon
● Communication with other work packages
● Task 5.1 ↔ WP4 & WP2
● Task 5.2 ↔ WP3
● Task 5.4 ↔ WP2
16. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
168 February 2019 meyer@astron.nl
WP5 Effort and Contacts
8 February 2019
17. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
178 February 2019 meyer@astron.nl
Discussions
8 February 2019
● WP5 F2F meeting
● How often?
● Co-organize with other WPs?
● WP5 hackathon
● The goal is to coordination and communication
across tasks in order to get things done
● 3-4 times per year
● Developers from WP5 and possibly other WPs at
one location
● Organize trainings together with other WPs to
demonstrate the science analysis platform
18. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
188 February 2019 meyer@astron.nl
WP5 workshop on Science
Platform design and requirements
8 February 2019
● When: 16-17 April 2019
● Where: ?
● More information will follow in the coming weeks
19. Funded by the European Union’s
Horizon 2020 - Grant N° 824064
8 February 2019 meyer@astron.nl
Thank you !