Rapid is a unique approach to quickly designing and delivering web portal interfaces for applications that require large amounts of computing resources or that need to run on specific servers. We will demonstrate the success of Rapid in a number of projects across a wide range of disciplines: brain imaging, chemistry, microscopy, engineering and seismology.
The Rapid approach consists of defining the resources, application use and user interface in one configuration file. This file is then validated and translated directly into a live portlet that can be inserted into a portal container. The whole process can be performed without any conventional programming. Rapid provides all the necessary components for handling compute-jobs. It knows how to handle remote files stores, monitor jobs, validate input, talk to Sun Grid Engine, Condor, PBS or just use a plain SSH connection.
This is the presentation I made at the 10th Annual Conference organised by the Paryavaran Dakshata Mandal at the Convocation Hall, University of Mumbai
Informações e dados sobre o lançamento do Programa Operação Chuvas de Verão 2016/2017 da cidade de Santo André. O POCV é coordenado pela Defesa Civil andreense através do Semasa (Serviço Municipal de Saneamento Ambiental de Santo André)
Softening agents are applied to textiles to improve their hand, drape, cutting and sewing qualities. An effective softener must be readily dispersible in rinse water and rapidly absorbed so that uniform deposition on the fabric can occur within a relatively short treatment time and generally, exhaustion should take place in about 5min for the softener to be effective and economically usable. It must impart softness, fluffiness and lubricity to the treated cloth and reduce static build-up, especially in the case of hydrophobic fibers like cellulose acetate, nylon, polyester and acrylic fibers. These effects should be obtained without the loss of fabric whiteness or brightness, and then the treated fabric should retain its ability to absorb in subsequent use for drying the body (bath towels) or other surfaces. Fabric softener (also called fabric conditioner) is used to prevent static cling and make fabric softer. It is available as a liquid or as dryer sheets. Popular brand names include Downy (Lenor), Snuggle, Bounce, Comfort and Sta-Soft.I believe that, the knowledge of this report will help future carrier of every textile engineer.
A Spring Batch bootcamp! Spring Batch is the open source batch processing framework from SpringSource, makes of the Spring framework. http://www.springsource.org/spring-batch
Need(le) for Speed - Effective Unit Testing for Java EEhwilming
Writing good concise tests for application components can be quite cumbersome. Especially, when they have a lot of dependencies to other beans and you do not want to manually write mock objects to fulfill all of them. On the other hand, you do not want to run too many slow integration tests.
We still see the need for a lightweight framework that allows to test components in isolation where their dependencies are resolved in an arbitrary manner. Sometimes you might want to use mock objects for dependencies, sometimes you would like to link your component to the real world.
With the Needle framework you can accomplish those goals in very comfortable way. Objects under test will get their dependencies injected automatically. The Needle core is extensible, it allows you to add your own injection points at run time. You are free to supply implementations yourself or rely on the mock objects provided by Needle. Needle will thus minimize the setup code and maximize the speed of a unit test.
Spring Day | Behind the Scenes at Spring Batch | Dave SyerJAX London
2011-10-31 | 01:30 PM - 02:15 PM
Spring Batch has a large user base and a good track record in production systems, but what is it all really about, and why does it work? This presentation provides a short bootstrap to get a new user started with the Batch domain, showing the key concepts and explaining the benefits of the framework. Then it goes into a deeper dive and looks at what holds it all together, with a close look at some of the most important but least understood features, including restart, retry and transactions.
Arnaud vous propose de découvrir le framework Spring Batch: du Hello World! jusqu'à l'exécution multi-threadée de batch, en passant par la lecture de fichiers CSV et la reprise sur erreur. Les techniques qu'utilise le framework pour lire et écrire efficacement de grands volumes de données e vous seront pas non plus épargnées ! La présentation se base sur une approche problème/solution, avec de nombreux exemples de code et des démos. A la suite de cette présentation, vous saurez si Spring Batch convient à vos problématiques et aurez toutes les cartes en mains pour l'intégrer à vos applications batch.
This is the presentation I made at the 10th Annual Conference organised by the Paryavaran Dakshata Mandal at the Convocation Hall, University of Mumbai
Informações e dados sobre o lançamento do Programa Operação Chuvas de Verão 2016/2017 da cidade de Santo André. O POCV é coordenado pela Defesa Civil andreense através do Semasa (Serviço Municipal de Saneamento Ambiental de Santo André)
Softening agents are applied to textiles to improve their hand, drape, cutting and sewing qualities. An effective softener must be readily dispersible in rinse water and rapidly absorbed so that uniform deposition on the fabric can occur within a relatively short treatment time and generally, exhaustion should take place in about 5min for the softener to be effective and economically usable. It must impart softness, fluffiness and lubricity to the treated cloth and reduce static build-up, especially in the case of hydrophobic fibers like cellulose acetate, nylon, polyester and acrylic fibers. These effects should be obtained without the loss of fabric whiteness or brightness, and then the treated fabric should retain its ability to absorb in subsequent use for drying the body (bath towels) or other surfaces. Fabric softener (also called fabric conditioner) is used to prevent static cling and make fabric softer. It is available as a liquid or as dryer sheets. Popular brand names include Downy (Lenor), Snuggle, Bounce, Comfort and Sta-Soft.I believe that, the knowledge of this report will help future carrier of every textile engineer.
A Spring Batch bootcamp! Spring Batch is the open source batch processing framework from SpringSource, makes of the Spring framework. http://www.springsource.org/spring-batch
Need(le) for Speed - Effective Unit Testing for Java EEhwilming
Writing good concise tests for application components can be quite cumbersome. Especially, when they have a lot of dependencies to other beans and you do not want to manually write mock objects to fulfill all of them. On the other hand, you do not want to run too many slow integration tests.
We still see the need for a lightweight framework that allows to test components in isolation where their dependencies are resolved in an arbitrary manner. Sometimes you might want to use mock objects for dependencies, sometimes you would like to link your component to the real world.
With the Needle framework you can accomplish those goals in very comfortable way. Objects under test will get their dependencies injected automatically. The Needle core is extensible, it allows you to add your own injection points at run time. You are free to supply implementations yourself or rely on the mock objects provided by Needle. Needle will thus minimize the setup code and maximize the speed of a unit test.
Spring Day | Behind the Scenes at Spring Batch | Dave SyerJAX London
2011-10-31 | 01:30 PM - 02:15 PM
Spring Batch has a large user base and a good track record in production systems, but what is it all really about, and why does it work? This presentation provides a short bootstrap to get a new user started with the Batch domain, showing the key concepts and explaining the benefits of the framework. Then it goes into a deeper dive and looks at what holds it all together, with a close look at some of the most important but least understood features, including restart, retry and transactions.
Arnaud vous propose de découvrir le framework Spring Batch: du Hello World! jusqu'à l'exécution multi-threadée de batch, en passant par la lecture de fichiers CSV et la reprise sur erreur. Les techniques qu'utilise le framework pour lire et écrire efficacement de grands volumes de données e vous seront pas non plus épargnées ! La présentation se base sur une approche problème/solution, avec de nombreux exemples de code et des démos. A la suite de cette présentation, vous saurez si Spring Batch convient à vos problématiques et aurez toutes les cartes en mains pour l'intégrer à vos applications batch.
Analyzing source code of WPF examples by the Infragistics CompanyPVS-Studio
We continue analyzing various C# projects in order to show the abilities of the static code analyzer, PVS-Studio. In this article, we are providing analysis results of WPF examples by the Infragistics Company. Infragistics is a major software vendor, founded in 1989. It gained popularity mainly through the development of enterprise-ready user interface toolsets for developers, which are run on all platforms, including .NET.
Executing hundreds or thousands of process instances per second? Yes, it's possible. This webinar is about best practices for high-load situations, and how to scale Camunda BPM horizontally.
Cloudera Sessions - Clinic 3 - Advanced Steps - Fast-track Development for ET...Cloudera, Inc.
Working with Hadoop does not always mean starting from scratch. In this session, you’ll learn how to leverage your existing investments in tools and skills to accelerate your Hadoop development. Learn from experts as they walk you step-by-step through the conversion of an existing ETL process to use Big Data.
Similar to Rapid Giving Computational Science A Friendly Face (20)
Edinburgh Data-Intensive Research Data-intensive refers to huge volumes of data, complex patterns of data integration and analysis, and intricate interactions between data and users. Current methods and tools are failing to address data-intensive challenges effectively. They fail for several reasons, all of which are aspects of scalability. The deluge of computational methods and plethora of computational systems prevents effective and efficient use of resources, user interfaces are not adopted at a sufficient rate to satisfy demand for scientific computing and data and knowledge is created outside suitable contexts for collaborative research to be effective. The Edinburgh Data-Intensive Research group addresses these scalability issues by providing mappings from abstract formulations to concrete and optimised executions of research challenges, by developing intuitive interfaces to enable access to steer these executions and by developing systems to aid in creating new research challenges. In this talk I will present several exemplars where we have dealt with scalability issues in scientific scenarios.
Science is witnessing a data revolution. Data are now created by faster and cheaper physical technologies, software tools and digital collaborations. Examples of these include satellite networks, simulation models and social network data. To transform these data successfully into information then into knowledge and finally into wisdom, we need new forms of computational thinking. These may be enabled by building "instruments" that make data comprehensible for the "naked mind" in a similar fashion to the way in which telescopes reveal the universe to the naked eye. These new instruments must be grounded in well-founded principles to ensure they have the fidelity and capacity to transform the complex and large-scale data into comprehensive forms; this demands new data-intensive methods.
Data-intensive refers to huge volumes of data, complex patterns of data integration and analysis and intricate interactions between data and users. Current methods and tools are failing to address data-intensive challenges effectively: they fail for several reasons, all of which are aspects of scalability. I will introduce three main aspects of data-intensive research and show how we are addressing the challenges that arise from the interaction of these aspects. I will make use of results from our interdisciplinary collaborations as examples of solutions to specific challenges that can arise when scaling up intensity.
Presentation for the Digital Repositories e-Science Network to introduce the new JISC-funded project which aims to deliver Google Maps for Developmental Biology.
10. JSDL Application Repository /
Job Submission Portal (JSR-168 Compliant)
My Grid Identity: None Help / Get Started | Logout
General portal
Authenticate Submit My Job
Applications
My Job : Weka "Instances" Test Save SaveAsNew New
Job Categories
Browse_Host
Review and submit My Job (the active job) to the selected candidate host.
Data_Transfer
Admin
My Job Summary
Info
• Far too many boxes to My Job:
Application Name: (Browse / Select New Application)
Application Version:
Weka
3.4.9
Edit
Edit
fill in Job Name: Weka "Instances" Test Edit
PageLayout
Job Category: Analysis/Stats Edit
JsdlLayout Selected Execute / Submission Endpoint: ngs.rl.ac.uk:2119/lsf Edit
Candidate Execution / Submission Endpoints: <Candidate Hosts> Update Edit
<JobIdentification/>
Executable: /usr/ngs/WEKA_3_4_9 Edit
JobName
Standard Input File: Edit
• Arcane technical JobDescription
JobProject
Standard Output File:
Standard Error File:
my_vsl_res
StdErr.txt
Edit
Edit
content required <Application/>
ApplicationName
Working Directory:
Status (Check): UNSUBMITTED
Edit
ApplicationVersion JobHandle:
Description Process Count: 1 Edit
Job Type: single Edit
<Posix/>
Node Count:
• Basically a command-
Edit
Executable Edit
Min Mem (MB):
Arguments Max Mem (MB): Edit
line / XML-editor in Input
Output
Arguments: weka.core.Instances soybean.arff Edit
disguise Error Job Description: Edit
Working Directory WEKA 3.4.9
Environment -------------------------------------------------------
template configured by J.Churchill (HPCSG,RAL,STFC)
Wall Time
updated by A.Maniopoulou (HPCSG,RAL,STFC)
<Resources/> -------------------------------------------------------
Candidate Hosts This is an simple "Instances" job. Weka is a serial code.
File Systems
The input files required for this example can be retrieved and staged into your home (or working) directory from
<DataStaging/> ngs.rl.ac.uk:/apps/weka/weka-3-4-9/data/soybean.arff. This template is setup to stage the example input files from this
directory. Before running this example, change and/or create the working directory on the "Active Job Profile Detail" page, then
Stage Data go to the 'StageData' page and click on 'StageNow' button to copy the files into that working directory.
Files/Links
To now run this example job (or another job based on this template) go to the "Submit" page and click on "Validate the RSL"
<JSDL/> before submission. If that gives no errors, check the box next to re-submit and then click the "Submit active job". The status of
Submit/Run the job should appear in red. First it will say submitted. To update the status, click on the status wording. When completed, go to
Environment Variables (None): Edit
File Systems: Edit
File System Name MountPoint Type
WORKINGDIR normal
USRNGS /usr/ngs normal
19. Welcome , Jano van Hemert Administration Content Layout Profile Home Logout
Simplicity of one XML file
Home
A Rapid Portlet
Filter an Image on the NGS
Choose Image
Portal Host
Manchester NGS Node
Oxford NGS Node
File System:
Leeds NGS Node
Pub-155-ftp
Pub-155
/..
Link
badelepht.jpg
paint.jpg
unique.jpg
Image:
Image Filename: /home/fileuser/Images/paint.jpg
Review and Submit Job
27 November 2007
20. Taskflow-based approach
Welcome , Jano van Hemert Administration Content Layout Profile Home Logout
Home
A Rapid Portlet
Filter an Image on the NGS
Choose Image
Portal Host
Manchester NGS Node
File System: Oxford NGS Node
Leeds NGS Node
Pub-155-ftp
Pub-155
/..
Link
badelepht.jpg
paint.jpg
unique.jpg
Image:
Image Filename: /home/fileuser/Images/paint.jpg
Review and Submit Job
27 November 2007
21. Taskflow-based approach
Welcome , Jano van Hemert Administration Content Layout Profile Home Logout
Home
A Rapid Portlet
Filter an Image on the NGS
Choose Image
Portal Host
Manchester NGS Node
File System: Oxford NGS Node
Leeds NGS Node
Pub-155-ftp
Pub-155
/..
Link
badelepht.jpg
paint.jpg
unique.jpg
Image:
Image Filename: /home/fileuser/Images/paint.jpg
Review and Submit Job
27 November 2007
Welcome , Jano van Hemert Administration Content Layout Profile Home Logout
Home
A Rapid Portlet
Filter Job:
Filter Input Image
Pub-155
emboss /home/fileuser/Images/paint.jpg
Submit to Host
Manchester
Leeds
Oxford
Fork
Back Submit Job
27 November 2007
22. What we offer
•Free software: all developed under Open Source licenses
•JSR-168 compliant portlets: you choose your favourite portal container
•Robust software: XML validation, FindBugs™, JUnit tests
•Extensive documentation
•Tutorials: to lower the learning curve
•Support from us: we want you to spread the good word!
23. Advanced features
• MPI Jobs
• Persistence between sessions
• Parameterised jobs
• Jython plugins
• Programmable buttons
30. research.nesc.ac.uk
Jano van Hemert & Malcolm Atkinson
Research Assistants
Jos Koetsier
Liangxiu Han
David Rodriguez
Gagarine Yaikhom
Laura Valkonen
PhD Students
Thomas French
Luna De Ferrari
Rob Kitchen
Yin Chen
Chee-Sun Liew
Fan Zhu
Editor's Notes
Script I wrote and executed on eddie or more formally known as ECDF
This is fine for the power user (physicist, astronomer, computer scientist)
In general, biologists, medical, social scientists, etc. are not comfortable with command lines, ssh logins, etc.
Only reserves compute time
Cannot actually define hobs
Portal for Nano-CMOS project that enables running simulations for engineers
Customer gets exactly what they ask for
Long development time (8 months training + 2 months coding)
What happens if developer disappears?
Custom build portals to cater for user-defined tasks
Hide the gory XML bits and job submission jargon with friendly user interface
Roll out job submission portals quickly, easy to build prototypes (no coding in Java!)
Also, these are JSP pages, they can be added to existing portals