Rapid Giving Computational Science A Friendly Face
•
1 like•662 views
Rapid is a unique approach to quickly designing and delivering web portal interfaces for applications that require large amounts of computing resources or that need to run on specific servers. We will demonstrate the success of Rapid in a number of projects across a wide range of disciplines: brain imaging, chemistry, microscopy, engineering and seismology. The Rapid approach consists of defining the resources, application use and user interface in one configuration file. This file is then validated and translated directly into a live portlet that can be inserted into a portal container. The whole process can be performed without any conventional programming. Rapid provides all the necessary components for handling compute-jobs. It knows how to handle remote files stores, monitor jobs, validate input, talk to Sun Grid Engine, Condor, PBS or just use a plain SSH connection.
Recommended
More Related Content
Viewers also liked
Viewers also liked (11)
Similar to Rapid Giving Computational Science A Friendly Face
Similar to Rapid Giving Computational Science A Friendly Face (20)
More from Jano van Hemert
Rapid Giving Computational Science A Friendly Face
- 1. Jano van Hemert and Jos Koetsier research.nesc.ac.uk/rapid NI VER U S E IT TH Y O F H G E R D I U N B
- 7. Computational science: portals the ultimate solution!
- 8. Computational science: portals the ultimate solution!
- 9. Ridiculous portal ! "#$%&'!()*!+,#-!!.!!/#0'!1(2! ! source: GENIUS portal guide
- 10. JSDL Application Repository / Job Submission Portal (JSR-168 Compliant) My Grid Identity: None Help / Get Started | Logout General portal Authenticate Submit My Job Applications My Job : Weka "Instances" Test Save SaveAsNew New Job Categories Browse_Host Review and submit My Job (the active job) to the selected candidate host. Data_Transfer Admin My Job Summary Info • Far too many boxes to My Job: Application Name: (Browse / Select New Application) Application Version: Weka 3.4.9 Edit Edit fill in Job Name: Weka "Instances" Test Edit PageLayout Job Category: Analysis/Stats Edit JsdlLayout Selected Execute / Submission Endpoint: ngs.rl.ac.uk:2119/lsf Edit Candidate Execution / Submission Endpoints: <Candidate Hosts> Update Edit <JobIdentification/> Executable: /usr/ngs/WEKA_3_4_9 Edit JobName Standard Input File: Edit • Arcane technical JobDescription JobProject Standard Output File: Standard Error File: my_vsl_res StdErr.txt Edit Edit content required <Application/> ApplicationName Working Directory: Status (Check): UNSUBMITTED Edit ApplicationVersion JobHandle: Description Process Count: 1 Edit Job Type: single Edit <Posix/> Node Count: • Basically a command- Edit Executable Edit Min Mem (MB): Arguments Max Mem (MB): Edit line / XML-editor in Input Output Arguments: weka.core.Instances soybean.arff Edit disguise Error Job Description: Edit Working Directory WEKA 3.4.9 Environment ------------------------------------------------------- template configured by J.Churchill (HPCSG,RAL,STFC) Wall Time updated by A.Maniopoulou (HPCSG,RAL,STFC) <Resources/> ------------------------------------------------------- Candidate Hosts This is an simple "Instances" job. Weka is a serial code. File Systems The input files required for this example can be retrieved and staged into your home (or working) directory from <DataStaging/> ngs.rl.ac.uk:/apps/weka/weka-3-4-9/data/soybean.arff. This template is setup to stage the example input files from this directory. Before running this example, change and/or create the working directory on the "Active Job Profile Detail" page, then Stage Data go to the 'StageData' page and click on 'StageNow' button to copy the files into that working directory. Files/Links To now run this example job (or another job based on this template) go to the "Submit" page and click on "Validate the RSL" <JSDL/> before submission. If that gives no errors, check the box next to re-submit and then click the "Submit active job". The status of Submit/Run the job should appear in red. First it will say submitted. To update the status, click on the status wording. When completed, go to Environment Variables (None): Edit File Systems: Edit File System Name MountPoint Type WORKINGDIR normal USRNGS /usr/ngs normal
- 11. Not about one portal for all
- 12. Not about one portal for all
- 13. Not about one portal for all
- 15. GridQTL !
- 16. Rapid
- 17. students researcher teacher 2. uses 3. reads 5. performs task 4. deploys portal designer web portal 1 .creates 6. runs jobs XML 7. returns interface, results !"#$%&"'( task & resource description compute resources
- 18. Simplicity of one XML file
- 19. Welcome , Jano van Hemert Administration Content Layout Profile Home Logout Simplicity of one XML file Home A Rapid Portlet Filter an Image on the NGS Choose Image Portal Host Manchester NGS Node Oxford NGS Node File System: Leeds NGS Node Pub-155-ftp Pub-155 /.. Link badelepht.jpg paint.jpg unique.jpg Image: Image Filename: /home/fileuser/Images/paint.jpg Review and Submit Job 27 November 2007
- 20. Taskflow-based approach Welcome , Jano van Hemert Administration Content Layout Profile Home Logout Home A Rapid Portlet Filter an Image on the NGS Choose Image Portal Host Manchester NGS Node File System: Oxford NGS Node Leeds NGS Node Pub-155-ftp Pub-155 /.. Link badelepht.jpg paint.jpg unique.jpg Image: Image Filename: /home/fileuser/Images/paint.jpg Review and Submit Job 27 November 2007
- 21. Taskflow-based approach Welcome , Jano van Hemert Administration Content Layout Profile Home Logout Home A Rapid Portlet Filter an Image on the NGS Choose Image Portal Host Manchester NGS Node File System: Oxford NGS Node Leeds NGS Node Pub-155-ftp Pub-155 /.. Link badelepht.jpg paint.jpg unique.jpg Image: Image Filename: /home/fileuser/Images/paint.jpg Review and Submit Job 27 November 2007 Welcome , Jano van Hemert Administration Content Layout Profile Home Logout Home A Rapid Portlet Filter Job: Filter Input Image Pub-155 emboss /home/fileuser/Images/paint.jpg Submit to Host Manchester Leeds Oxford Fork Back Submit Job 27 November 2007
- 22. What we offer •Free software: all developed under Open Source licenses •JSR-168 compliant portlets: you choose your favourite portal container •Robust software: XML validation, FindBugs™, JUnit tests •Extensive documentation •Tutorials: to lower the learning curve •Support from us: we want you to spread the good word!
- 23. Advanced features • MPI Jobs • Persistence between sessions • Parameterised jobs • Jython plugins • Programmable buttons
- 24. Define Compute Resources • Condor / Sun Grid Engine / PBS / GridSAM / Fork process
- 25. Why choose Rapid? • Focuses on users’ tasks • Delivers solutions quickly • Empowers your group • Lowers development costs • Lowers maintenance effort
- 26. Use Cases
- 30. research.nesc.ac.uk Jano van Hemert & Malcolm Atkinson Research Assistants Jos Koetsier Liangxiu Han David Rodriguez Gagarine Yaikhom Laura Valkonen PhD Students Thomas French Luna De Ferrari Rob Kitchen Yin Chen Chee-Sun Liew Fan Zhu
Editor's Notes
- Script I wrote and executed on eddie or more formally known as ECDF This is fine for the power user (physicist, astronomer, computer scientist) In general, biologists, medical, social scientists, etc. are not comfortable with command lines, ssh logins, etc.
- Only reserves compute time Cannot actually define hobs
- Portal for Nano-CMOS project that enables running simulations for engineers Customer gets exactly what they ask for Long development time (8 months training + 2 months coding) What happens if developer disappears?
- Custom build portals to cater for user-defined tasks Hide the gory XML bits and job submission jargon with friendly user interface Roll out job submission portals quickly, easy to build prototypes (no coding in Java!) Also, these are JSP pages, they can be added to existing portals