EGI-ACE’s main goal is to implement the compute platform of the European Open Science Cloud and contribute to the EOSC Data Commons by delivering integrated computing platforms, data spaces and tools as an integrated solution that is aligned with major European cloud federation projects and HPC initiatives.
This presentation introduces you to the architecture and composition of the EOSC Compute Platform, which delivers capabilities at the IaaS, PaaS and SaaS level.
The EOSC Compute Platform with the EGI-ACE project
1. www.egi.eu
@EGI_eInfra
The work of the EGI Foundation
is partly funded by the European Commission
under H2020 Framework Programme
EGI: Advanced Computing for Research
EGI-ACE
The EOSC Compute Platform
2. @EGI_eInfra
www.egi.eu 04/02/2021 2
EGI-ACE Mission
Implement the Compute Platform of the European Open
Science Cloud and contribute to the EOSC Data Commons
by delivering integrated computing, platforms, data spaces
and tools as an integrated solution that is aligned with major
European cloud federation projects and HPC initiatives.
3. @EGI_eInfra
www.egi.eu 04/02/2021 3
“EOSC needs to provide a long-term
open data archive with high-
performance storage and computing
services, to enable sustainable use of
data beyond the life span of individual
data infrastructures” (Feb 2020)
How EGI-ACE responds to the EOSC mission?
https://zenodo.org/record/3675081
4. @EGI_eInfra
www.egi.eu 04/02/2021 4
EGI-ACE specific objectives
Objective 1. Deliver the European Open Science Cloud Compute Platform and
expand the supply-side
Objective 2: Contribute to the implementation of the EU Data Strategy
and the EOSC Data Commons to support the Green Deal, Health and
Fundamental Research
Objective 3: Integrate the EOSC Compute Platform with the EOSC
Portal and the EOSC Core
Objective 4: Contribute to the realization of a global Open Science Cloud
Objective 5: Expand the demand-side of EOSC across sectors and disciplines
EOSC-Exchange
EGI-ACE
Federated
Data
EOSC
Core
O1
O2
O3
O4
O5
Solutions for a sustainable EOSC
report from the EOSC Sustainability WG
5. @EGI_eInfra
www.egi.eu 04/02/2021 5
[WP5] EOSC Compute
Platform (SaaS) – Thematic
Data Analytics, Simulation
Tools and Data
EGI-ACE and the EOSC Architecture
• EOSC-Core: “assembles all the
basic elements to operate and
provide the means to discover,
share, access and reuse data
and services in a reliable
manner”
• EOSC-Exchange: “builds on the
EOSC-Core to ensure that a
rich set of services (common
and thematic), exploiting FAIR
data and encouraging its reuse,
are available to publicly funded
researchers”
[WP3-4-6] EOSC
Compute Platform
(IaaS/PaaS) - Common
6. @EGI_eInfra
www.egi.eu 04/02/2021 6
EGI-ACE concept and methodology:
Tier service architecture
Data Spaces
and Analytics
Data and thematic
data analytics and
processing tools
Platforms
generic added-value
platform level
services
Federated
Access
Federation-wide
management of
data and computing
Federated
Resources
Compute and
storage facilities
Interactive
Computing
Distributed AI
training
Workload
Management
Automated Cluster
deployment
HPC
Public
Clouds
HTC
Research
Clouds
Federated Identity
Federated Compute Federated Data
Fundamental
Science Data
Space
Health Data
Space
Green Deal
Data Space
EOSC
portal
EOSC
users
Service
Management,
Tools,
Processes,
Policies
Humanities
Data Space
Early
adopters
7. @EGI_eInfra
www.egi.eu 04/02/2021 7
EGI federation - Core tools
Service Management, Tools, Processes,
Policies
Interactive
Computing
Distributed AI
training
Workload
Management
Automated Cluster
deployment
HPC
Public
Clouds
HTC
Research
Clouds
Federated Identity
Federated Compute Federated Data
Fundamental
Science Data
Space
Health Data
Space
Green Deal
Data Space
EOSC
portal
EOSC
users
Humanities
Data Space
Early
adopters
IT Service Management
Accounting Monitoring Helpdesk
Check-in
Configuration DB Operations
portal
Unified
middleware
8. @EGI_eInfra
www.egi.eu 04/02/2021 8
EGI-ACE Consortium Overview
• EGI Foundation as coordinator
▪ 33 participating partners, 23 third-parties
• Consortium at a glance
▪ 16 partners to deliver cloud, HTC and HPC resources
▪ 12 user access and platform solution providers
▪ 21 providers from 13 communities to deliver data
spaces
▪ 12 federation service providers
+ Early adopters; Partner infrastructures
57% of the budget for service provisioning
9. @EGI_eInfra
www.egi.eu 04/02/2021 9
For users and communities within the project AND for new EOSC users
• Increasing capacity (Cloud and HTC):
▪ Phase 1 (month 1-10) ⇒ 13.8 Million CPU hours/41,300 GPU hours/7.5 PB/month
▪ Phase 2 (month 11-20) ⇒ 27.6 Million CPU hours/83,000 GPU hours/15 PB/month
▪ Phase 3 (month 21-30) ⇒ 41.4 Million CPU hours/124,000 GPU hours/23 PB/month
• Mix of funding mechanisms:
▪ Virtual Access:
▪ Local grants enabling policy-based access (EGI Cloud Federation + new partners):
▪ Pay-for-use:
▪ Pilot HPC sites:
Infrastructure layer – Cloud, HTC, HPC
Moldova Hungary
Georgia
Latvia Austria USA China South Africa
Diverse locations
AND
compute flavors
10. @EGI_eInfra
www.egi.eu 04/02/2021 10
Resource allocation, user support (2/2)
Single users
Small groups
Experimental users
International projects
Multi-national communities
● Bring the user to ‘fit for purpose’ services
● Bring data to user (DataHub)
● Bring application to user (AppDB)
● Aligned customer relationship mgm.
National
resources
(more than EGI-ACE)
User profiling
Project focus on the providers
New community
Virtual
Organisations
Call evaluation
● Select suitable providers
● Select suitable services
● Deploy, configure, test
● Support scale-up and uptake
Project focus on the users
OneData
EC3
VMOps Notebooks
Existing
Virtual
Organisations
(more than EGI-ACE)
DIRAC
DODAS
DEEP training
INDIGO PaaS
...
11. @EGI_eInfra
www.egi.eu 04/02/2021 11
13 Data
spaces
OPERAS
metrics and
certification
service
WeNMR
(structural
biology)
Virtual
Imaging
Platform
(medical
imaging)
OpenRiskNet
NanoCommons
UseGalaxy.eu
(bioinformatics)
OPENCoastS
(circulation
forecast)
ENES
(climate
analytics)
PROMINENCE
(fusion
physics)
LOFAR
science
products
(astronomy)
SeaDataNet
WebOcean
Data Analysis
EMSO ERIC
data services
GBIF Cloud
data space
Disaster
mitigation and
agriculture
Health
and
medicine
Climate
research
Energy and physical
sciences
Environmental
sciences
Humanities
Earthquake
response and
landslide
analysis
E-RIHS
(heritage
object
analytics)
VIRGO
(gravitational
waves)
PHIRI
(population
health)
MeerKAT
workflows
(astronomy)
GEOSS
Portal and
GEO DAB
EISCAT_3D
Data Portal
(atmospheric
physics)
7 Early adopters
12. @EGI_eInfra
www.egi.eu 04/02/2021 12
EOSC Exchange Capacity Building
INFRAEOSC-07
Increasing the
service offer of
the EOSC Portal
Distributed
Computing
Services
Data discovery,
deposition, bit
preservation
storage
Open
scholarship
services
C-SCALE
(Copernicus
EOSC data
analytics
engine)
RELIANCE
(Research
Lifecycle
Management
for Copernicus
users)
Collaboration agreements
14. @EGI_eInfra
www.egi.eu 04/02/2021 14
INFRAEOSC-03 H2020 Call
• Architecture Contribute to
▪ The interoperability framework
▪ Relevant task forces
• EOSC Portal Provide requirements
• Support and training
▪ Open calls for business pilots and research projects
▪ Research Infrastructure Science Projects
▪ Coordinate resource allocation, provide technical support and resources
▪ Coordinate delivery of training and provide training resources
• Collaboration Coordinate with EOSC projects, initiatives and the EOSC
AISBL
EGI-ACE and EOSC Future