This document discusses using cloud computing for bioinformatics. It begins by defining cloud computing and describing its key characteristics like on-demand access to computing resources and rapid elasticity. It then discusses different cloud delivery models like Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS). The document provides examples of public cloud providers for each delivery model. It also introduces tools like CloudBridge that help make applications cloud-independent and CloudLaunch, a portal for deploying cloud-enabled bioinformatics applications. Finally, it briefly discusses how these tools and cloud resources can help improve bioinformatics workflows by providing scalable infrastructure for processing large genomic datasets.

1. Cloud Computing and
Bioinformatics
Enis Afgan*, Nuwan Goonasekera†
* Johns Hopkins University, Taylor Lab, USA
†
University of Melbourne, Victorian Life Science Computation Initiative, Australia
@ University of Colombo
Feb 2017

2. Overview
• The key characteristics of cloud computing
• Dynamically scaling cloud resources
• Using Cloud Computing for bioinformatics
Source: http://dilbert.com/strips/comic/2012-05-25/

3. Life before cloud computing
source: http://www.rackspace.com/knowledge_center/whitepaper/revolution-not-evolution-how-cloud-computing-differs-from-traditional-it-and-why-it

4. Cloud Computing: A Definition
• NIST definition: “Cloud computing is a model for enabling
ubiquitous, convenient, on-demand network access to a
shared pool of configurable computing resources (e.g.,
networks, servers, storage, applications, and services) that
can be rapidly provisioned and released with minimal
management effort or service provider interaction.”
» National Institute of Standards and Technology
(http://csrc.nist.gov/publications/nistpubs/800-145/SP800-145.pdf)

5. The Cloud Model
Private Community Public Hybrid
Deployment
Models
Delivery
Models
Essential
Characteristics
Software as a Service
(SaaS)
Platform as a Service
(PaaS)
Infrastructure as a
Service (IaaS)
• On-demand self-service
• Broad network access
• Resource pooling
• Rapid elasticity
• Measured service

6. Delivery Models
source: http://www.businessinsider.com.au/10-most-important-in-cloud-computing-2013-4?op=1#a-word-about-clouds-1

7. Public SaaS examples
• Gmail
• Sharepoint
• Salesforce.com CRM
• On-live
• Gaikai
• Microsoft Office 365
• Some definitions include those that do not require payment.
E.g. ad-supported sites

8. Public PaaS Examples
Cloud Name Language and
Developer Tools
Programming
Models Supported
by Provider
Target Applications
and Storage Options
Google App Engine Python, Java, Go,
PHP + JVM languages
(scala, groovy, jruby)
MapReduce, Web,
DataStore, Storage
and other APIs
Web applications and
BigTable storage
Salesforce.com’s
Force.com
Apex, Eclipsed-based
IDE, web-based
wizard
Workflow, excel-like
formula, web
programming
Business applications
such as CRM
Microsoft Azure .NET, Visual Studio,
Azure tools
Unrestricted model Enterprise and web
apps
Amazon Elastic
MapReduce
Hive, Pig, Java, Ruby
etc.
MapReduce Data processing and
e-commerce
Aneka .NET, stand-alone
SDK
Threads, task,
MapReduce
.NET enterprise
applications, HPC

9. Infrastructure-as-a-Service (IaaS)
• Amazon Web Services (Market leader)
• Rackspace Cloud
• NeCTAR/OpenStack Research Cloud
• Joyent Cloud
• GoGrid
• FlexiScale

10. Common Terms
Machine Image: A stored image/template from which a new virtual
machine can be launched. E.g. Ubuntu
Instance: A running virtual machine based on some machine image.
Volume: Attachable Block Storage, which is the equivalent of a virtual disk
drive.
Object Store: A large store for storing simple binary objects + metadata
within containers
Security Groups: A means of specifying firewall rules
Key-pairs: Public/private key pairs for accessing a virtual machine

11. Getting started with Cloud resources

12. Demo 1

13. Many clouds exist - how do we use them?

14. Many clouds and many solutions
launch.genome.edu.au ; use.jetstream-cloud.org ; launch.usegalaxy.org
?!?!

15. Architectural stack
CloudLaunch.usegalaxy.org
A P P L I C A T I O N S
CloudBridge
CloudMan
Goonasekera, N., Lonie, A., Taylor, J., Afgan, E., “CloudBridge – a Simple Cross-Cloud Python
Library”, XSEDE 16, Miami, FL, July 2016.

16. Demo 2

17. CloudBridge Design Principles
A simple, open-source python multi-cloud library.
Uniform API irrespective of the underlying provider
No special casing of application code
Simpler code
Provide a set of conformance tests for all supported clouds
No need to test against each cloud
“Write-once-run-anywhere”
> 92% test coverage at present
Supports OpenStack and AWS right now
Community contributions for GCE and Azure forthcoming!
http://cloudbridge.readthedocs.org/
https://github.com/gvlproject/cloudbridge

18. Sample code: launch an instance
1. kp = provider.security.key_pairs.create('cloudbridge_intro')
2. with open('cloudbridge_intro.pem', 'w') as f:
3. f.write(kp.material)
4. sg = provider.security.security_groups.create(
5. 'cloudbridge_intro', 'A security group used by CloudBridge')
6. sg.add_rule('tcp', 22, 22, '0.0.0.0/0')
7. img = provider.compute.images.get(image_id)
8. inst_type = sorted([t for t in provider.compute.instance_types.list() if t.vcpus >= 2 and t.ram >= 4],
key=lambda x: x.vcpus*x.ram)[0]
9. inst = provider.compute.instances.create(
name='CloudBridge-intro', image=img, instance_type=inst_type,
key_pair=kp, security_groups=[sg])
10. # Wait until ready
11. inst.wait_till_ready()
12. # Show instance state
13. inst.state
14. # 'running'
15. inst.public_ips
16. # [u'54.166.125.219']
Create a key pair
Create a security group
Launch an instance

19. Portal for deploying cloud-enable applications.
Support for customization
Support launch for diff versions, apps, configs, clouds → fill a role of a science
gateway discovery and access portal
Modular and extensible platform
App-store for cloud-enabled applications
Users can develop and integrate custom application launch and management
components, at the UI and backend
Natively multi-cloud
Backed by CloudBridge
CloudLaunch Feature Highlights
https://beta.launch.usegalaxy.org/
https://github.com/galaxyproject/cloudlaunch-ui
https://github.com/galaxyproject/cloudlaunch

20. CloudLaunch architecture
CloudBridge
Django + REST framework + Celery
Angular 2
GVL
CloudMan
Galaxy
CloudMan
Ubuntu Pluggable
components

21. Pluggable component example
<form class="form" [ngFormModel]="gvlLaunchForm" (ngSubmit)="onSubmit(gvlLaunchForm.value)">
<!-- GVL Component Selection -->
<config-panel>
<panel-header>GVL Settings</panel-header>
<panel-body>
<div class="form-group">
<label>Auto-start the selected applications</label>
<div class="checkbox">
<label>
<input type="checkbox" name="gvlapp_cmdlineutils" ngControl="gvl_cmdline_utilities" />
GVL Commandline Utilities </label>
</div>
<div class="checkbox">
<label>
<input type="checkbox" name="gvlapp_smrt_analysis" ngControl="smrt_portal" />
PacBio SMRT Analysis </label>
</div>
</div>
</panel-body>
</config-panel>
<!-- CloudMan settings -->
<cloudman-config [initialConfig]="initialConfig.config_gvl"></cloudman-config>
</form>

22. Cloud capacity is great -
but what do we use it for?

23. Bioinformatics: in one slide
A multi-disciplinary science using computers for acquiring, managing and
analyzing biological data.
It is a data-driven science.
Biology Medicine
Math &
Physics
Computer
Science
Bioinformatics

24. What type of data are we talking about?
DNA → RNA → Protein → to Complex… to Tissues… to Organs… to full
Organisms
Each cell contains an (almost) the same DNA in it nuclei.
Adult human body has approximately 37 trillion cells.

25. Apply data transformations to extract useful information
This is not always a well-defined process
This is typically done with existing tools, or by developing one’s own
Tools can be chained into workflows
What do we do with the data?

26. And how do we obtain such data?
First methods developed in the mid-1970’s, called Sanger sequencing.
In the 1990’s, the international Human Genome Project took 13 years to sequence
the human genome.
In the 2000’s, massively parallel Next Generation Sequencers (NGS) were
developed that took days to sequence a human genome at a much lesser cost.
Today, nanopore sequencers are emerging offering real time sequencing.
There are many public data repositories with free access to data (e.g., TCGA,
1000 genomes, GenBank).

27. omicsmaps.com

30. Results
External reference
data
Raw
data
Raw data to results
100-1000's GB
few GB
Typical genomics flow

31. Results
Raw
data
Some computers + reliable persistent data storage +
bioinf tools + reference data + workflow system
100-1000's GB
few GB
Indexed
genomes
10-100's GB
Aug
Sep
Oct
Nov
...
A real-world system

32. A Data analysis and integration tool
A (free for everyone) web service integrating a
wealth of tools, compute resources, terabytes of
reference data and permanent storage
Open source software that makes integrating your
own tools and data and customizing for your own
site simple

33. Galaxy: accessible analysis system

34. Three ways to use Galaxy
1. Download and run locally
2. Public website (http://usegalaxy.org)
3. Run on the Cloud

35. 100sG
B
100+

36. Results
Raw
data
Some computers + reliable persistent data storage +
bioinf tools + reference data + workflow system
100-1000's GB
few GB
Indexed
genomes
10-100's GB
Aug
Sep
Oct
Nov
...
A real-world system

37. CloudBridge
CloudLaunch
CloudMan
CloudScale
Pathway Expected Outcomes
Improved features (root volume size)
Cloud independence
Improved stability
Federated single-sign on
“One-click” launch
Bulk launch
Cloud Independence
Tasks
Complete
Use CloudBridge
Assemble image from
Docker containers
Remove shared filesystem
Simpler deployment
Extensible platform
Scaling for institutional Galaxy instances
Scale-out support for labs
Audience
All users
Academic users
All
Virtual labs
(e.g., GVL,
CLIMB)
CLIMB
/Other labs
Hosted services
Tutorials
Complete
Progress roadmap

38. Acknowledgments

39. Did this sound interesting?
This entire project is an effort from a large community!
Come talk to us - get involved.
enis.afgan@jhu.edu or nuwan.goonasekera@unimelb.edu.au