Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Webinar on Galaxy & Galaxy integration with the Dataminer Service in the context of AGINFRA+ Project

65 views

Published on

Galaxy is a scientific workflow, data integration, and data and analysis persistence and publishing platform that aims to make computational biology accessible to research scientists that do not have computer programming or systems administration experience. AGINFRA+ VREs include access to a Galaxy deployment where researchers can make use of the usual Galaxy features and also interact with the DataMiner platform to run the available algorithms.

The webinar introduced Galaxy, the available deployment for AGINFRA+ VREs and the DataMiner integration.

Τhe webinar was described by Enol Fernández from EGI Advanced Computing Services for Research.

Published in: Software
  • Be the first to comment

  • Be the first to like this

Webinar on Galaxy & Galaxy integration with the Dataminer Service in the context of AGINFRA+ Project

  1. 1. WWW.PLUS.AGINFRA.EU AGINFRA PLUS - Accelerating user-driven e-infrastructure innovation in Food & Agriculture has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 731001. Galaxy for AGINFRA+ VREs Online webinar | January 2019 Enol Fernández EGI Foundation
  2. 2. WWW.PLUS.AGINFRA.EU Webinar is being recorded! Galaxy for AGINFRA+ VREs 2
  3. 3. WWW.PLUS.AGINFRA.EU Demonstrate how scientific communities working on agriculture and food topics may carry out rapid and intuitive development and deployment of innovative applications and workflows, powered by open e-infrastructures. Strengthen and illustrate the value and potential of AGINFRA+ as a virtual research environment for the domain of agriculture and food. The AGINFRA+ Vision Galaxy for AGINFRA+ VREs 3
  4. 4. WWW.PLUS.AGINFRA.EU Enol Fernández  Cloud Architect @ EGI Foundation  Working since 2003 on computing e-infrastructures (grid, cloud)  Support to user communities  EGI Cloud Federation Coordinator  Development of new services for EGI Galaxy for AGINFRA+ VREs 4 /usr/bin/whoami
  5. 5. WWW.PLUS.AGINFRA.EU Galaxy Galaxy integration in AGINFRA+ Demo Q&A Outline Galaxy for AGINFRA+ VREs 5
  6. 6. WWW.PLUS.AGINFRA.EU Galaxy 6
  7. 7. WWW.PLUS.AGINFRA.EU Web-based platform for computational biomedical research (analysis and data integration) Open source, community driven software that makes integrating your biomed tools simple Galaxy for AGINFRA+ VREs 7
  8. 8. WWW.PLUS.AGINFRA.EU Galaxy is "an open, web-based platform for performing accessible, reproducible, and transparent genomic science.” Accessibility  Users without programming experience can easily upload/retrieve data, run complex tools and workflows, and visualize data Reproducibility  Galaxy captures information so that any user can understand and repeat a complete computational analysis Transparency  Users can share or publish their analyses (histories, workflows, visualizations) Galaxy for AGINFRA+ VREs 8 Galaxy: core values
  9. 9. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 9 Galaxy interface Toolbar HistoryMain
  10. 10. WWW.PLUS.AGINFRA.EU Tools manipulate and transform your data Each tool is defined by:  input datasets, parameters, commands, and outputs  help, tests, citations, dependency requirements New versions can be installed without removing old ones to ensure reproducibility Galaxy Tool Shed: application store with +1000 tolos ready to be used Galaxy for AGINFRA+ VREs 10 Tools
  11. 11. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 11
  12. 12. WWW.PLUS.AGINFRA.EU Every action and dataset is tracked on the history  Datasets produced by tools  Operations performed on data Includes:  name, format, size, creation time, datatype-specific metadata  tool id, version, inputs, parameters  standard output (stdout) and error (stderr)  state (waiting, running, success, failed)  hidden, deleted, purged Galaxy for AGINFRA+ VREs 12 History
  13. 13. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 13 History management
  14. 14. WWW.PLUS.AGINFRA.EU Series of tools and dataset actions that run in sequence as a batch operation  From history, from scratch with visual editor or import Allow to re-run the same analysis on different input data sets  Change parameters before re-running a similar analysis Workflows can be annotated, viewed, shared, published, and imported - just like other Galaxy objects. Galaxy for AGINFRA+ VREs 14 Workflows
  15. 15. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 15 Workflow editor Image taken from https://mgescan.readthedocs.io/en/latest/workflow.html
  16. 16. WWW.PLUS.AGINFRA.EU Galaxy on AGINFRA+ 16
  17. 17. WWW.PLUS.AGINFRA.EU Workflow design and execution technologies are key for the research activities of the use cases Two platforms selected:  KNIME  Galaxy Integration with AGINFRA+:  Access the workflow platforms within the VRE  Access other VRE tools form the workflow platforms Galaxy for AGINFRA+ VREs 17 Workflows on AGINFRA+
  18. 18. WWW.PLUS.AGINFRA.EU Custom wsgi filter for Galaxy  Leverages support for external authentication in Galaxy  Exposes REMOTE_USER with information from D4Science  Stores the D4Science token to further interaction with the VRE Galaxy for AGINFRA+ VREs 18 Authentication
  19. 19. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 19
  20. 20. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 20 DataMiner and Galaxy Periodic synchronisation of DataMiner as tools in Galaxy Execution using WPS calls to Dataminer  With user token  Use DataMiner algorithm as any other Galaxy tool
  21. 21. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 21
  22. 22. WWW.PLUS.AGINFRA.EUGalaxy for AGINFRA+ VREs 22 Some limitations Input files must be provided as URLs  No easy way to discover the files from the Galaxy interface Output is stored as HTML with pointers to the actual outputs  Can still be easily converted to other more friendly format as needed  A csv extractor provided to facilitate that case, more can be created.
  23. 23. WWW.PLUS.AGINFRA.EU Kubernetes Cluster (EGI Cloud Container Compute) Galaxy for AGINFRA+ VREs 23 Deployment SSL Certificate Kubernetes Ingress AuthN filter WPS calls
  24. 24. WWW.PLUS.AGINFRA.EU DEMO! 24
  25. 25. WWW.PLUS.AGINFRA.EU Galaxy available now for AGINFRA+ VREs  Go and test! Start collecting your feedback:  Add new tools as needed by use cases  Improve the integration with DataMiner (input and output files)  Expand deployment as usage grows Galaxy for AGINFRA+ VREs 25 Next steps
  26. 26. WWW.PLUS.AGINFRA.EU Galaxy: https://galaxyproject.org/ Galaxy Tutorials: https://galaxyproject.org/learn/ Galaxy training: https://galaxyproject.github.io/training-material/ Galaxy ToolShed: https://toolshed.g2.bx.psu.edu/ DataMiner: https://wiki.gcube- system.org/gcube/DataMiner_Manager AGINFRA+ VREs: https://aginfra.d4science.org/ Galaxy for AGINFRA+ VREs 26 Reference
  27. 27. WWW.PLUS.AGINFRA.EU CONSORTIUM WWW.PLUS.AGINFRA.EU Thanks! Questions? @enolfc https://linkedin.com/in/enolfc/ Galaxy for AGINFRA+ VREs 27

×