We highlight our efforts to leverage many of these existing solutions into an integrated system that makes the installation, life-cycle management and scaling of Galaxy uniform across a variety of platforms (e.g., workstations, clusters, clouds). This is accomplished by leveraging existing Ansible playbooks and Docker containers and integrate them with Kubernetes Helm charts, CloudLaunch, and CloudMan. This will allow changes with a few clicks through a GUI, as well as via a command line client or an API. Gradually, this Galaxy management system will lead to a mostly “hands-off” approach to running Galaxy, with support for rolling updates and zero-downtime configuration changes.

1. From laptop to super-computer:
standardizing installation and
management of Galaxy
Nuwan Goonasekera, Enis Afgan, Vahid Jalili
June 27, 2018
GCCBOSC 2018; Portland, OR

2. Basics are simple
getgalaxy.org
$ git clone -b release_18.05 https://github.com/galaxyproject/galaxy.git
$ sh run.sh
...
http://localhost:8080
$ docker run -d -p 8080:80 -p 8021:21 -p 8022:22 bgruening/galaxy-stable
...
http://localhost:8080
https://launch.usegalaxy.org/ → Genomics Virtual Lab (GVL) / CloudMan

3. For production systems, software stack grows
Robust
database
Job
manager
FTP
server
Shared data External
resources

4. Updating settings
$ vi $GALAXY_ROOT/config/galaxy.yml
...
# admin_users: your@email.com
# brand: My Galaxy
...
$ <ctrl+c> && sh run.sh
18.01
18.05
18.09
...

5. High-quality deployments require attention
Monitoring
and alerts
Upgrades
and backups
Access control and institutional
auth system integration

6. 100+ public
Galaxy Servers
bit.ly/gxyServers
World-wide, that’s a lot of repeated effort to maintain generally similar systems.

7. Standardizing Galaxy deployments: what?
Create one stack for deploying Galaxy.
Make it repurposable, multi-platform, upgradeable, configurable, codified,
consistent, extensible.

8. Standardizing Galaxy deployments: why?

9. - Consolidate best deployment practices: production setting from the start
- Highly automated process: DevOps-based
- Build on top of successful and proven technologies: Docker & Kubernetes
- Get extra features
- More intuitive configuration changes
- Built-in upgrades: developed by the project to apply on other installations (i.e., like DB migrations)
- No downtime restarts
- Self-healing
- Monitoring
- Support for a variety of infrastructures
- Extensible via container image extension
- At the expense of conforming to some system minimums: Docker and/or
Kubernetes
Standardizing Galaxy deployments: how?
Do for installing Galaxy what Conda did for installing tools.

10. Demo - three times over

11. - Marcus Christie, Sue Gyoung Kim,
Purva Sane
- RC Carter, Elizabeth Bruce, Alex Lenail
- Matt Bookman, Bai Zhang,
Ehsan Chiniforooshan, Sheila Reynolds
- Pablo Moreno
- Alex Mahmoud
Acknowledgments
Tomorrow (Thu): 12:30 - 13:30

12. If more time...

13. Bootstrap via
CloudLaunch
>_ run
VM IP
CloudBridge
AWS Azure GCE OpenStack
CloudLaunch-plugin
galaxy/cloudman-bootcloudman-boot → Rancher K8S Helm
CloudMan
chart CloudBridge CloudLaunch CloudMan HelmsMan
Multi-cloud Infrastructure Coordination Applications
VM
...
...
...
...
External app
repositories
GrafanaKeycloak
Chart Chart
Galaxy Postgres RabbitMQ ProFTP
Galaxy Chart
ansible-galaxy, ansible-galaxy-extras, ...

14. Ongoing and future work
We’d like feedback about your scenarios and environments.
- In active development:
- Infrastructure scaling, including auto-scaling
- Persistent storage
- Image design → faster startup → horizontal Galaxy scaling → automated upgrades & rollbacks
- Integration with existing resources
- Infrastructure bursting, Pulsar integration
- Auth (e.g., InCommon)
- UI
- GCE support
- Documentation

15. Takeaway messages
- Everything you can do via the UI, you can do via the API and CLI
(hint: think bursting)
- Kubernetes is awesome
- Everything is customizable
- Everything is generalizable
- Learn everything once: from training to production, it’s the same system