Your SlideShare is downloading. ×
F03-Cloud-Obiwee
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

F03-Cloud-Obiwee

407
views

Published on

OBIWEE: An open source bioinformatics cloud environment (Jonathan Piat)

OBIWEE: An open source bioinformatics cloud environment (Jonathan Piat)

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
407
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. OBIWEE : an open source bioinformatics cloud environmentOBIWEE : On Demand Bioinformatics Intensive Workflow Execution Environment J. Piat, F. Moreews, O. Sallou http://vapor.gforge.inria.fr/ OBIWEE - BOSC 2011, July 16, Vienna
  • 2. What is OBIWEE?OBIWEE is an open source bioinformatics IntensiveComputation Execution environment based on SLICEE.Preconfigured on a scalable linux virtual cluster withTorgue job scheduler, it can be deployed on a privatecloud, using OpenNebula, or EC2 public cloud.S3 is used as a persistent storage layer (Eucalyptus Walrusor Amazon S3).Based on Ubuntu/Debian linux bioinformaticsimages/packages. OBIWEE - BOSC 2011, July 16, Vienna
  • 3. What is OBIWEE?1/ A workflow authoring tool +2/ A virtual cluster (Torque) +3/ A set of deployment scripts for Private cloud (OpenNebula / KVM ) and/or Public cloud (EC2) OBIWEE - BOSC 2011, July 16, Vienna
  • 4. OBIWEE : components1/ SLICEE : A workflow authoring tool● Tools description is command-line based: Write the command line as on local, in your workflow, execute on remote All installed tools immediately available Easy file referencing method● Job scheduler front end (queue selection per job)● Set of reference ID : dataset reference mechanism for remote serviceinvocation● Access data via URI : multiple protocols (sftp,ftp,http,file,s3) +internal ref. ID URI.● Standard authentication(ssh)● Persistence and logs● Automatic coarse grain parallelism extraction: Basic bioinformatics formats implemented Easy extension with regular expressions/external scripts OBIWEE - BOSC 2011, July 16, Vienna
  • 5. OBIWEE : components 2/ A virtual clusterA scalable cluster using Torque/SGE schedulerWorkflow jobs and parallelized jobs are submitted to theDRM manager. It is easy to scale the DRM to increase theworkload capacity of the tool. OBIWEE - BOSC 2011, July 16, Vienna
  • 6. OBIWEE : components3/ A set of deployment scripts for Private cloud (OpenNebula / KVM ) and/or Public cloud (EC2) OBIWEE - BOSC 2011, July 16, Vienna
  • 7. OBIWEE : installationVirtual image creation: Bioinformatics software needs Lyncee installation install needs Image configuration needs provides Slicee install provides Cluster generationWorkflow management NFS mount of working directoryData parallelization Node deploymentData managementJob submissionAuthorization OBIWEE - BOSC 2011, July 16, Vienna
  • 8. OBIWEE : architecture nodeClient NFS share(Kepler/command line) Run job node Amazon EC2/ master Open Nebula Add node node publish Retrieve Amazon EC2/ S3 Open Nebula OBIWEE - BOSC 2011, July 16, Vienna
  • 9. OBIWEE : clients● API (job submission) : create your own submission/orchestration clients CommonRestClient client = new CommonRestClient(serverUrl); //upload data client.upload(sessionId, inputDataUriPath); //asynchronous execution rdsid = client.getDSIDFromAsyncExe(xmlQuery, sessionId); //wait (client.waitAndGetResult()),or do something else //download /move results client.move(vaporSession, uri,new URI(path));● Command line (workflow execution) java -cp $cp vapor.cli.VaporCmdClient -w workflow.xml -i input.xml -d auth.xml● GUI (workflow execution and design): Kepler with SLICEE actors (workflow creation/execution) OBIWEE - BOSC 2011, July 16, Vienna
  • 10. OBIWEE : KEPLER client with SLICEE actors miRNAs detection workflow OBIWEE - BOSC 2011, July 16, Vienna
  • 11. OBIWEE : road map Road map ● Monitoring, fail over ● Custom full web client ● Integration in existing popular clients ● Data cleanup policies THANK YOU !more info on SLICEE and OBIWEE EC2 deployment tutorial at http://vapor.gforge.inria.fr/ OBIWEE - BOSC 2011, July 16, Vienna