Reproducible bioinformatics for everyone: Nextflow & nf-core

•

0 likes•370 views

Slides from my talk at the Karolinska Institute in Huddinge (Stockholm, Sweden). June 2022. General introduction to Nextflow and nf-core, covering what they are and why you should use them!

Science

Phil Ewels

phil@seqera.io
Reproducible bioinformatics for everyone:
 
Nextflow & nf-core

SGE
PBS
LSF
Singularity
Microsoft Azure
Language

Singularity
AWS CodeCommit
SGE
PBS
LSF
Microsoft Azure
Azure Repos

A community e
ff
ort to collect a curated set
of analysis pipelines built using Next
fl
ow.
https://nf-co.re

https://nf-co.re
Guidelines Tools Pipelines

https://nf-co.re
Develop with
the community
Use a common
template
Collaborate,
don’t duplicate

https://nf-co.re
37

RELEASED
24

UNDER DEVELOPMENT
6

ARCHIVED

Documentation
Slack workspace
Hackathons
Twitter updates
Starter template
Code guidelines
Helper tools
CI code linting

and tests
Stable pipelines
Centralised

con
fi
gs
Download
 
for o
ffl
ine use
List and update

pipelines

Launch wizard
nf-core launch --id 1637063024_d92e0f1632c2
nextflow run nf-core/rnaseq -params-file nf-params.json
https://nf-co.re/launch

Running in the cloud https://github.com/nf-core/viralrecon

Running in the cloud https://nf-co.re/viralrecon

Next
fl
ow DSL2
Modular design gives
clearer pipeline code
Di
ff
erent pipelines can
reuse tool wrappers
and software images
Proper unit testing of individual
steps in each pipeline
Library of tool wrappers makes
building a new pipeline fast

Next
fl
ow DSL2
Update command
fetches latest versions
CI tests check that module
fi
les have not been edited
Metadata
fi
le tracks git
hash of modules repo
Copies module
fi
les in
to pipeline
$ nf-core modules install fastqc
$ nf-core modules list

Join the community
https://nf-co.re/join

Intuitive launchpad interface
Launch, manage, and monitor
Share runs and work in teams
Create cloud infrastructure with a click
https://tower.nf

https://nf-co.re/join
Phil Ewels
https://phil.ewels.co.uk

phil@seqera.io
https://seqera.io

What's hot

DPDK & Cloud NativeMichelle Holley

BPF & Cilium - Turning Linux into a Microservices-aware Operating SystemThomas Graf

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs

Developing real-time data pipelines with Spring and Kafkamarius_bogoevici

No instrumentation Golang Logging with eBPF (GoSF talk 11/11/20)Zain Asgar

Patterns of evolution from monolith to microservicesKarina Mora

Understanding eBPF in a Hurry!Ray Jenkins

IPMI is dead, Long live RedfishBruno Cornec

Linux Profiling at NetflixBrendan Gregg

Advanced TerraformSamsung Electronics

eBPF/XDP Netronome

Automating AWS Infrastructure Provisioning Using Concourse and TerraformCesar Rodriguez

Introduction to eBPFRogerColl2

eBPF - Rethinking the Linux KernelThomas Graf

BPF - in-kernel virtual machineAlexei Starovoitov

Debian or Yocto Project? Which is the best for your Embedded Linux project?Chris Simmonds

Apache Flink Training: DataStream API Part 1 BasicFlink Forward

Terraform 0.9 + good practicesRadek Simko

VPNaaS in NeutronKazunori Takeuchi

Scale Kubernetes to support 50000 servicesLinuxCon ContainerCon CloudOpen China

What's hot (20)

DPDK & Cloud Native

BPF & Cilium - Turning Linux into a Microservices-aware Operating System

Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...

Developing real-time data pipelines with Spring and Kafka

No instrumentation Golang Logging with eBPF (GoSF talk 11/11/20)

Patterns of evolution from monolith to microservices

Understanding eBPF in a Hurry!

IPMI is dead, Long live Redfish

Linux Profiling at Netflix

Advanced Terraform

eBPF/XDP

Automating AWS Infrastructure Provisioning Using Concourse and Terraform

Introduction to eBPF

eBPF - Rethinking the Linux Kernel

BPF - in-kernel virtual machine

Debian or Yocto Project? Which is the best for your Embedded Linux project?

Apache Flink Training: DataStream API Part 1 Basic

Terraform 0.9 + good practices

VPNaaS in Neutron

Scale Kubernetes to support 50000 services

Similar to Reproducible bioinformatics for everyone: Nextflow & nf-core

Nextflow Camp 2019: nf-core tutorialPhil Ewels

nf-core: A community-driven collection of omics portable pipelinesJose Espinosa-Carrasco

Opnfv vision, community and projectsOPNFV

nf-core usage tutorialGisela Gabernet

AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...Amazon Web Services

Tutorial contributing to nf-coreGisela Gabernet

ApacheCon 2021 - Apache NiFi Deep Dive 300Timothy Spann

DevOps Fest 2020. Даніель Яворович. Data pipelines: building an efficient ins...DevOps_Fest

Summit 16: Keynote: HPE Presentation- Transforming Communication Service Prov...OPNFV

Introduction of eBPF - 時下最夯的Linux Technology Jace Liang

Sql bits apache nifi 101 Introduction and best practicesTimothy Spann

Redfish and python-redfish for Software Defined InfrastructureBruno Cornec

ApacheCon 2021: Apache NiFi 101- introduction and best practicesTimothy Spann

ELIXIR Proteomics Community - Connection with nf-corePhil Ewels

Common design patterns (migang 16 May 2012)Steven Smith

OpenCL Overview Japan Virtual Open House Feb 2021The Khronos Group Inc.

Alfresco Day Platform Updateohej

Data Plane and VNF Acceleration Mini Summit Open-NFP

Pottnet Meetup Essen - ASP.Net CoreMalte Lantin

Pottnet MeetUp Essen - ASP.Net CoreMalte Lantin

Similar to Reproducible bioinformatics for everyone: Nextflow & nf-core (20)

Nextflow Camp 2019: nf-core tutorial

nf-core: A community-driven collection of omics portable pipelines

Opnfv vision, community and projects

nf-core usage tutorial

AWS re:Invent 2016: Open Source at AWS—Contributions, Support, and Engagement...

Tutorial contributing to nf-core

ApacheCon 2021 - Apache NiFi Deep Dive 300

DevOps Fest 2020. Даніель Яворович. Data pipelines: building an efficient ins...

Summit 16: Keynote: HPE Presentation- Transforming Communication Service Prov...

Introduction of eBPF - 時下最夯的Linux Technology

Sql bits apache nifi 101 Introduction and best practices

Redfish and python-redfish for Software Defined Infrastructure

ApacheCon 2021: Apache NiFi 101- introduction and best practices

ELIXIR Proteomics Community - Connection with nf-core

Common design patterns (migang 16 May 2012)

OpenCL Overview Japan Virtual Open House Feb 2021

Alfresco Day Platform Update

Data Plane and VNF Acceleration Mini Summit

Pottnet Meetup Essen - ASP.Net Core

Pottnet MeetUp Essen - ASP.Net Core

Recently uploaded

Harmful and Useful Microorganisms Presentationtahreemzahra82

Speech, hearing, noise, intelligibility.pptxpriyankatabhane

Scheme-of-Work-Science-Stage-4 cambridge science.docxyaramohamed343013

Call Girls in Munirka Delhi 💯Call Us 🔝9953322196🔝 💯Escort.aasikanpl

Vision and reflection on Mining Software Repositories research in 2024AyushiRastogi48

Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCRlizamodels9

TOPIC 8 Temperature and Heat.pdf physicsssuserddc89b

Evidences of Evolution General Biology 2John Carlo Rollon

insect anatomy and insect body wall and their physiologyDrAnita Sharma

BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.PraveenaKalaiselvan1

Call Girls in Mayapuri Delhi 💯Call Us 🔝9953322196🔝 💯Escort.aasikanpl

Grafana in space: Monitoring Japan's SLIM moon lander in real timeSatoshi NAKAHIRA

‏‏VIRUS - 123455555555555555555555555555555555555555kikilily0909

TOTAL CHOLESTEROL (lipid profile test).pptxdharshini369nike

Solution chemistry, Moral and Normal solutionsHajira Mahmood

Transposable elements in prokaryotes.pptArshadWarsi13

THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptxNandakishor Bhaurao Deshmukh

zoogeography of pakistan.pptx fauna of Pakistanzohaibmir069

Forest laws, Indian forest laws, why they are importantadityabhardwaj282

Welcome to GFDL for Take Your Child To Work DayZachary Labe

Recently uploaded (20)

Harmful and Useful Microorganisms Presentation

Speech, hearing, noise, intelligibility.pptx

Scheme-of-Work-Science-Stage-4 cambridge science.docx

Call Girls in Munirka Delhi 💯Call Us 🔝9953322196🔝 💯Escort.

Vision and reflection on Mining Software Repositories research in 2024

Call Girls In Nihal Vihar Delhi ❤️8860477959 Looking Escorts In 24/7 Delhi NCR

TOPIC 8 Temperature and Heat.pdf physics

Evidences of Evolution General Biology 2

insect anatomy and insect body wall and their physiology

BIOETHICS IN RECOMBINANT DNA TECHNOLOGY.

Call Girls in Mayapuri Delhi 💯Call Us 🔝9953322196🔝 💯Escort.

Grafana in space: Monitoring Japan's SLIM moon lander in real time

‏‏VIRUS - 123455555555555555555555555555555555555555

TOTAL CHOLESTEROL (lipid profile test).pptx

Solution chemistry, Moral and Normal solutions

Transposable elements in prokaryotes.ppt

THE ROLE OF PHARMACOGNOSY IN TRADITIONAL AND MODERN SYSTEM OF MEDICINE.pptx

zoogeography of pakistan.pptx fauna of Pakistan

Forest laws, Indian forest laws, why they are important

Welcome to GFDL for Take Your Child To Work Day

Reproducible bioinformatics for everyone: Nextflow & nf-core

1. Phil Ewels phil@seqera.io Reproducible bioinformatics for everyone:   Nextflow & nf-core

3. Language

4. Processes Channels Workflows

6. Parallelisation Reentrancy Reusability

7. Software Compute Language

8. Singularity Compute Language

9. SGE PBS LSF Singularity Microsoft Azure Language

10. Singularity AWS CodeCommit SGE PBS LSF Microsoft Azure Azure Repos

11. Reproducible Portable

12.

13. A community e ff ort to collect a curated set of analysis pipelines built using Next fl ow. https://nf-co.re

14. https://nf-co.re Guidelines Tools Pipelines

15. https://nf-co.re Develop with the community Use a common template Collaborate, don’t duplicate

16. https://nf-co.re 37 RELEASED 24 UNDER DEVELOPMENT 6 ARCHIVED

17. https://nf-co.re 37 RELEASED 24 UNDER DEVELOPMENT 6 ARCHIVED

18. Community https://nf-co.re/stats

19. Community https://nf-co.re/community

20. Documentation Slack workspace Hackathons Twitter updates Starter template Code guidelines Helper tools CI code linting and tests Stable pipelines Centralised con fi gs Download   for o ffl ine use List and update pipelines

21. Documentation https://nf-co.re/rnaseq

22. Launch wizard https://nf-co.re/launch

23. Launch wizard nf-core launch --id 1637063024_d92e0f1632c2 nextflow run nf-core/rnaseq -params-file nf-params.json https://nf-co.re/launch

24. Running in the cloud Microsoft Azure

25. Running in the cloud https://github.com/nf-core/viralrecon

26. Running in the cloud https://nf-co.re/viralrecon

27. Running in the cloud https://nf-co.re/viralrecon

28. Next fl ow DSL2 Modular design gives clearer pipeline code Di ff erent pipelines can reuse tool wrappers and software images Proper unit testing of individual steps in each pipeline Library of tool wrappers makes building a new pipeline fast

29. Next fl ow DSL2 Update command fetches latest versions CI tests check that module fi les have not been edited Metadata fi le tracks git hash of modules repo Copies module fi les in to pipeline $ nf-core modules install fastqc $ nf-core modules list

30. Next fl ow DSL2

31. Next fl ow DSL2

32. Join the community https://nf-co.re/join

33. Intuitive launchpad interface Launch, manage, and monitor Share runs and work in teams Create cloud infrastructure with a click https://tower.nf

34. https://nf-co.re/join Phil Ewels https://phil.ewels.co.uk phil@seqera.io https://seqera.io