European Open Science Cloud: Concept, status and opportunities.
Presentation given by Gergely Sipos at the International Symposium on Grids and Clouds 2019 event in Taiwan.
Apidays Singapore 2024 - Building Digital Trust in a Digital Economy by Veron...
European Open Science Cloud: Concept, status and opportunities
1. EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536.
eosc-hub.eu
@EOSC_eu
European Open Science Cloud
Concept, status and opportunities
Gergely Sipos (EGI Foundation)
ISGC 2019, Taipei, April 2019
2. 2
European Open Science Cloud (EOSC) – A short history
EOSC-hub
- 4 areas
- 5 opportunities – with examples
Implementing an Open Science story
Next events
Outline
3. 3
EOSC - Problem statement
European Cloud Initiative
by the European Commission (April 2016)
1. How to maximise the incentives for sharing data and to
increase the capacity to exploit them?
2. How to ensure that data can be used as widely as
possible, across scientific disciplines and between the
public and the private sector?
3. How better to interconnect the existing and the new data
infrastructures across Europe?
4. How best to coordinate the support available to
European data infrastructures as they move towards
exascale computing?
“…a trusted, open environment for the scientific community for
storing, sharing and re- using scientific data and results…”
“…by 2020…”
5. 5
The federated infrastructure and supporting initiative providing
all researchers, innovators, companies and citizens
with seamless access to an open-by-default, efficient and
cross-disciplinary environment
for storing, accessing, reusing data, tools, publications and other
scientific outputs for research, innovation and educational purposes
About the European Open Science Cloud
Credits: EOSCpilot (https://eoscpilot.eu/)
6. Vision
100
partners
36
months
33
million
Euro
Researchers from all disciplines
have easy, integrated and open access to the
advanced digital services, scientific instruments,
data, knowledge and expertise they need
to collaborate to achieve excellence
in science, research and innovation
7. Mission
EGI
Federation
EUDAT
INDIGO-
DataCloud
Research
Infrastructur
es
The EOSC-hub project mobilises providers of
European relevance offering services, software and
data for advanced data-driven research and
innovation.
These resources are offered via the Hub – the
integration and management system of the
European Open Science Cloud, acting as a
European-level entry point for all stakeholders.
8. 8
The Compute Platform: EGI
Services for cloud, HTC, container computing and data staging
Based on NGIs – National compute centres – coordinated by EGI Foundation
4.4 Billion CPU core wall time in 2018, 1 M cores, > 730 PB
1170 open access publications, +41 new international projects
31 large scale ESFRI projects/landmarks supported
10. 10
The project established ‘The Hub’
https://marketplace.eosc-portal.eu
• Data
• Applications & tools
• Baseline services
(storage, compute,
connectivity)…
• Training, consultants
• Marketplace
• AAI
• Accounting
• Monitoring
• …
Usage according to
Rules of Participation
From the consortium AND from
external contributors
• Lightweight certifi-
cation of providers
• SLA negotiation
• Customer Relationship
Management
• …
Based on FitSM
• Security regulations,
• Compliance to standards,
• Terms of use,
• FAIR implementation
guidelines
• …
11. 11
Area 1: Services
e-Infra
EGI
Federation
EUDAT CDI
INDIGO-
DataCloud
Humanities
Language
and
literature
(CLARIN)
Arts
(DARIAH)
Engineering
Environmen
tal
engineering
(sea vessels,
LNEC)
Civil
Engineering
(Disaster
Mitigation)
Medical
and Health
Sciences
Biological
Sciences
(ELIXIR)
Structural
biology
(WeNMR)
Natural
sciences
PHYSICAL SCIENCES
Astronomy (LOFAR)
Fusion (ITER)
High Energy Physics
(CMS and VIRGO)
Space Science
(EISCAT-3D)
EARTH SCIENCE
EO Pillar
GEO
Climate Research
(ENES)
Seismology (ORFEUS,
EPOS)
BIOLOGICAL
SCIENCES
Marine and
freshwater biology
(IFREMER)
Biodiversity
conservation
(LifeWatch)
Ecology (ICOS)
Generic
services
+ a growing number of external contributions
12. 12
Generic services from e-Infrastructures:
Support to the Research Data Lifecycle
Processing & Analysis
Data Management, Curation &
Preservation
Access, Deposition & Sharing
● B2FIND (data)
● Marketplace (Services)
● Applications on Demand
● Federated HTC & Cloud Compute IaaS & PaaS
● Processing of sensitive data
● Jupyter Notebook
● Application DB (software & VM)
● B2DROP (data)
● B2Note (data)
● B2SHARE (data)
● DataHub
● B2HANDLE
● B2SAFE
● European Certified Trusted
Repository
● Thematic data analytics
● Scientific Workflow Management,
Orchestration (DIRAC, PaaS Orchestrator)
1
2
3
4
Discover & Reuse
13. 13
Area 2: Federation services
EOSC Portal
website and
Marketplace
Authentication-
Authorization
Infrastructure
Accounting
Monitoring
Helpdesk
Service
integration
support and
training
Engagement
with new
service
providers
Making
services
discoverable
and orderable
Making
services
accessible
Making
services
robust
Making
problems
resolvable
Making
services
usable
14. • IT service management
- Focuses on the provision of high quality IT services that meet customers' and users’
expectations
- Defines, documents and maintains service management processes with assigned
roles and responsibilities
• EOSC-hub Service Management: Based on FitSM standard
- 85 requirements that should be fulfilled by an organisation (or federation)
- 16 general requirements; 69 process-specific requirements
- 14 processes
14
Area 3: Federated operation
• Service portfolio management
• Service level management
• Incident management
• Change management
• Capacity management
• Information security management
• …
Example – Service Portfolio Management requirements:
PR1.1 A service portfolio shall be maintained. All services shall be specified
as part of the service portfolio.
PR1.2 Design and transition of new or changed services shall be planned.
PR1.3 Plans for the design and transition of new or changed services shall
consider timescales, responsibilities, new or changed technology,
communication and service acceptance criteria.
PR1.4 The organisational structure supporting the delivery of services shall be
identified, including a potential federation structure as well as contact points
for all parties involved.
• Implementation in EOSC
- EOSC-hub Federation services (will) implement all processes: First internal audit took place
- Other services are encouraged and supported in implementation/alignment
(FitSM training, consultancy, ‘implementation workshop’)
15. • Harmonising service-related policies and supporting providers in the
implementation
• Current focus
- Acceptable use policy
- Operations security policy
- Conditions of use
• Exploring new areas
- Licenses
- ‘FAIR’ services and data (Findable, Accessible, Interoperable, Reusable)
16
Area 4: Processes and policies
17. EOSC-hub Collaboration Areas: 5 opportunities
EOSC Digital Innovation Hub
Join as provider
Use of existing service
Networking
Training, user support
and best practices
18https://www.eosc-hub.eu/publications
18. 19
O1: Access services: https://eosc-portal.eu
Some services
accessible without
authentication
Some services work
with automated login
Some require
manual approval of
users
‘Simple’ requests:
passed to individual
service providers
Complex requests:
supported by
EOSC-hub tech
team
19. 21
O2: Join as provider - Benefits
• Promote adoption of your services
• Get a free online platform to manage the services request
and interact with users (Marketplace)
• Get statistics about access and customer feedbacks
• Contribute to the definition and maintenance of the EOSC
service provisioning policies and portfolio roadmap
(providers’ board)
20. 22
O2: Join as provider - Requirements
https://eosc-portal.eu/for-providers
• The service is accessible by users outside the original community
• The service is described via the common template (for publication on the
EOSC portal)
• At least one service instance is running in a production environment
(software vs. service)
• Published research data is ‘FAIR’
• Release notes and documentation are available
• Helpdesk channels are available 57 Services
published
+28 in
verification
52 service
providers
85 Services
until March
2019
Including
iCOMCOT and
WRF portals
from AS
21. 24
O3: EOSC Digital Innovation Hub (DIH)
• What are Digital Innovation Hubs?
• An ecosystem of start-ups, SMEs, large industries, researchers, accelerators and investors that fosters the
creation of partnerships to stimulate innovation.
• 200+ DIHs registered comprising both regional and European ones that focus on a variety of sectors and
topics
• The EOSC DIH
• To onboard industrial partnerships within the European Open Science Cloud
• Focused on (not limited to) start-ups and SMEs
• To persist beyond the life of the project
• Main activity: Setup and run ‘Business pilots’
Join the DIH
Propose/run a Business Pilot
22. Examples: Business pilots (initial 6)
ACTION Seaport
Space Weather Data for the DRACO
Observatory
Bot Mitigation Engine Sport Smart Video Analysis
CyberHAB
Furniture Enterprise Analytics Datafurn
Improving safety and operational
performance of seaports
A Cloud Framework for State-of-the-art
Space Weather
Using data cloud Services to manage
harmful algae blooms
Solutions to secure online services from
Botnets Attacks
Analyzing Sport Performance through a
Cloud-hosted platform
Platform-as-a-Service Data Analytics for the
Furniture Industry
• All defined architecture and mapped to EOSC-hub generic services
• Dedicated overview webpages for each pilot: https://eosc-hub.eu/digital-innovation-hub
25
23. EOSC-hub coordinates the development and delivery of EOSC related
trainings
- Service-specific training
- Training to facilitate Open Science – E.g. RDM good practices; IT Service
Management; …
Training resources online (courses and content)
- Innovation Management
- Service Management
- IT Security
- Data management
- EOSC services
26
O4: Training and support
https://www.eosc-hub.eu/training-material
Attend training
Become training provider
25. 28
Competence Centers - Use cases
Simplified use case Implemented by
Application and data portal with compute
and data management back-end
• EISCAT_3D
• ICOS-eLTER
Data and application replication backbone
for community centres
• ELIXIR
• EPOS-ORFEUS
‘Dropbox-like’ access to diverse and
scattered data
• Marine
User workflows over distributed data and
compute resources
• Fusion
• Radio astronomy
• Disaster Mitigation
26. 29
Uptake of generic services in the Comp. Centres
• CC leaders’ meetings
• 2 face-to-face meetings
• Monthly teleconference
• WP10 requirements DB:
• User stories
• Use cases
• Service requirements
GREY - considered for adoption, but the integration is yet to start
YELLOW - integration is ongoing
BLUE - technology is integrated and assessment is yet to finish
RED - technology was assessed and was found unsuitable
GREEN - technology is integrated and positively evaluated
27. 30
An Open Science story we can already implement with
EOSC
GitHub
Your
repository
EGI Notebooks
services
Zenodo
Your
laptop
Upload ipynb file
Add requirements.txt
Execute
Data repository
MyBinder.org
Re-execute
Obtain GitHub project reference
Provide GitHub project reference
Discover Notebook
(use DOI)
Fellow
researchers
Journal
paper
DOI
Notebooks
tutorial on
Tuesday
28. 31
An Open Science story we aim for with EOSC
GitHub
Your
repository
EGI Notebooks
and Binder service
Zenodo
Your
laptop
Execute
Obtain GitHub project reference
Provide GitHub project reference
Discover Notebook
(use DOI)
Fellow
researchers
Journal
paper
DOI
Distributed
big data
DataHub
B2DROP
Etc.
Data repository
Upload ipynb file
Add requirements.txt
29. EOSC-hub Week 2019
10-12 April, Prague
Highlights:
• Keynote by ESFRI Chair
• Science discipline specific sessions
• Plenary about ‘how to join’
• Service providers’ bootcamp
• FAIR services workshop with RDA and OpenAIRE
• Training (FitSM, Federated AAI, Data Management, Orchestration)
30. To engage with EGI service providers
(generic services in EOSC)
Conference (May 6-8)
Design Your e-Infrastructure workshop
(May 9):
• Intro to EGI services & analysis of science
community use cases
33
EGI Conference and
Design Your e-Infratructure workshop
