This document summarizes ELIXIR's plans to develop a cloud computing platform to support life science research across Europe. It discusses ELIXIR's goals to integrate user authentication, rationalize reference data distribution, support hybrid cloud/HPC deployments, develop a task distribution network using Kubernetes, and support workflow engines. Key components include Biocontainers for tools, RDSDS for reference data, TESK for task execution, and WES-ELIXIR for workflows. The platform aims to be compatible with GA4GH standards and support projects like EOSC-Hub and EOSC-Life.

1. eosc-hub.eu
@EOSC_eu
EOSC-hub receives funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 777536.
Dr Susheel Varma, EMBL-EBI
Dissemination level: Public

2. www.elixir-europe.org
EOSC-Hub: ELIXIR Competence Centre
Susheel Varma (susheel.varma@ebi.ac.uk)
ELIXIR Competence Centre Project Manager

3. ELIXIR Today
• ELIXIR has reached a healthy critical mass:
• International coordination through 23 Nodes
• Reaching over 200 institutes
• Joint scientific direction through 23 national node
directors (HoN committee)
• Coordinated ELIXIR infrastructure developments
(Platform ExCos)
• ELIXIR Communities validating infrastructure
services
• Strong implementation mechanisms:
• Legal framework for coordinated transnational
actions
• Experience operating large infrastructure grant
consortia

4. Data flowing freely between connected national infrastructures
Data
Tools
Standards
Compute
Training and
Skills
…delivered in partnership with
research communities...the
ELIXIR Communities

5. ELIXIR 2019-23 Programme
• Approved by ELIXIR Board in November 2018
• Came into effect 1 January 2019
• Accompanied by Financial Plan (2019-23)
• Programme coordinated from the ELIXIR Hub
and implemented through Annual Work
Plans via Commissioned Services
• Mid-term review of Programme planned for
2021
https://www.elixir-europe.org/about-us/what-we-do/elixir-programme

6. ELIXIR Strategic Objectives – 2019-2023 Work Programme

7. The changing landscape of life science research

8. ELIXIR Compute Platform for EOSC-Hub & EOSC-Life
• Integrate user federation into local compute
and data deployments - ELIXIR AAI
• Rationalise a ELIXIR-wide Data Distribution
Network – starting with Reference datasets ~
RDSDS
• Drive ELIXIR Compute Platform support to
Nodes to develop hybrid cloud/HPC
deployments in ELIXIR and EOSC
• Develop Task Distribution Network using Task
orchestration engines – e.g. Kubernetes ~ TESK
• Support national or regional Workflow
Choreography Engines – e.g. CWL-TES,
Cromwell, Nextflow, Galaxy, etc. ~ WES-ELIXIR

9. Developing standards and tools
Simplify the way people search for and request access
to potentially identifiable data in international and
national genomic data resources
Working towards GA4GH standards, APIs and toolkits to be used
throughout ELIXIR Nodes for human data discovery and access

10. ELIXIR Cloud and AAI Programme
WES
Workflow Execution Service
DRS
Data Repository Service
TRS
Tool Registry Service
TES
Task Execution Service
RDSDSTESKWES-ELIXIRBiocontainers
AAI
ELIXIR TOOLS
PLATFORM ELIXIR COMPUTE PLATFORM

11. ELIXIR Human Data – RD-Connect Pipeline
• ELIXIR Human Data – Rare Disease
Community
• Demonstrator focussed on the variant-calling pipeline
curated by the RD-Connect platfom authors – Laurie et
al 2016
• RD researchers submit raw (fastq) files from EGA and
data transfers
• The platform scales the analysis to obtain unannotated
gVCF files
• Further analysis via RD-Connect by data transfer back
to RD-Connect
• Pipeline converted to CWL and Nextflow for execution
via the platform

12. Marine Microbial Eukaryote Transcriptome Sequencing Project (MMETSP)
• 2018-IS-Interop-CWL
• Pipeline for the assembly and annotation of
marine eukaryotic transcriptome
• Datasets: Tara Ocean and MMETSP
• Goal is to develop and deploy CWL workflows to
multiple environments
• ELIXIR GA4GH Compatible Platform (WES - TES)
• ELIXIR Galaxy Community (usegalaxy.eu)
• Current focus on:
• Containerisation of the assembly and annotation
pipelines
• Galaxy – CWL Interoperability
• Ready for multi-site CWL execution

13. ELIXIR Marine: Marine Metagenomics (MMG) Workflow

14. ELIXIR Cloud Analysis Platform – Current Status
AuthN
AuthZ
Embass
y
cloud
Biocontainers 2.0
WES-ELIXIR
CWL-TES
ELIXIR-CH & CSC (cPouta)
TESK
FTP
EMBL-EBI (Embassy)
TESK
ELIXIR-FI (Rahti)
ELIXIR-IT
ELIXIR-
DE
ELIXIR-
CH

15. GA4GH-ELIXIR TRS Services (2018-IS-Tools-Biocontainers 2.0)

16. Reference Data Set Distribution Service (RDSDS)
• Developed as part of EOSC-Hub
• RDSDS – Data Distribution Network
• Provides a centralised Dataset registry,
management and distribution
• Integrated with ELIXIR AAI for AuthN-Z
• Supports HTTP, FTP, GSIFTP, S3, FTS3 and Globus
Connect
• Decentralised Identifiers using content-based
dataset identifiers
• Virtual Dataset sidecar container for local data
cache (Q2 2019)
• EMBL-EBI Data Archives (>80% Reference Data)
• Metabolights, ArrayExpress, PRIDE, ENA*

17. RDSDS Dataset Management Lifecycle

18. RDSDS Architecture

19. Upgrading RDSDS --> GA4GH DTS v1.0 Compatibility
• Ref Dataset Distribution Service
• DRS Compatibility API
• Support for:
• Databundles à Datasets
• WIP for:
• Dataobjects à Files
• Std Development:
• Data Transfers
• Data Lifecyle Mgmt.
• Subscriptions
• 2017-Data-Movement
• EUDAT 2020
• EOSC-Hub

21. ELIXIR-GA4GH Compatible Cloud Ecosystem
Biocontainers 2.0
- 6000+ tools
- MMG & MMETSP wfs
- Std development
http://api.biocontainers.pro/swagger-
ui.html#/Tools
TRS
WES-ELIXIR
- Refactored Kirini [Prototype]
- Support for TES Backend
- ELIXIR AAI Integration
- CWL-TES integration
- [WIP] Cromwell & Nextflow*
http://193.167.189.73:7777/ga4gh/wes/v1/ui/
WES
TESK
- Wrapper around K8s
- ELIXIR AAI
- [WIP] ResUtil dev
- EOSC Deploy. & AAI
https://tesk.c01.k8s-popup.csc.fi
https://tes.tsi.ebi.ac.uk
http://cloud-90-147-75-83.cloud.ba.infn.it:8080/
TES
RDSDS
- [WIP] DRS
Compatibility
Wrapper API
https://dsds-service.ebi.ac.uk
DRS

22. ELIXIR Compute Platform for EOSC-Hub & EOSC-Life
• Integrate user federation into local compute
and data deployments - ELIXIR AAI
• Rationalise a ELIXIR-wide Data Distribution
Network – starting with Reference datasets ~
RDSDS
• Drive ELIXIR Compute Platform support to
Nodes to develop hybrid cloud/HPC
deployments across ELIXIR and EOSC
• Develop Task Distribution Network using Task
orchestration engines – e.g. Kubernetes ~ TESK
• Support national or regional Workflow
Choreography Engines – e.g. CWL-TES,
Cromwell, Nextflow, Galaxy, etc. ~ WES-ELIXIR

26. Thank you to Heads of
Nodes, Platform and
Communities leaders,
Training and Technical
Coordinators and the whole
ELIXIR Community!