SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our User Agreement and Privacy Policy.
SlideShare uses cookies to improve functionality and performance, and to provide you with relevant advertising. If you continue browsing the site, you agree to the use of cookies on this website. See our Privacy Policy and User Agreement for details.
Successfully reported this slideshow.
Activate your 30 day free trial to unlock unlimited reading.
Good Practices for Developing Scientific Software Frameworks: The WRENCH framework example (and some others)
What is the best way to write secure and reliable applications?
- Good Enough Practices in Scientific Computing
(https://arxiv.org/abs/1609.00037)
- Best Practices for Scientific Computing (https://doi.org/10.1371/journal.pbio.1001745)
- Scientific Software Best Practices (https://scientific-software-best-practices.readthedocs.io/en/latest/)
- Best Practices for Scientific Software (https://software.ac.uk/blog/2017-11-29-best-practices-scientific-software)
Transcript
1.
Good Practices for Developing
Scientific Software Frameworks
The WRENCH framework example (and some others)
Rafael Ferreira da Silva
https://rafaelsilva.com
https://wrench-project.org
In collaboration with:
2.
What is the best way to write
secure and reliable applications?
2
5.
Good Enough Practices in Scientific Computing
• Among several recommendations, this paper emphasizes the following
aspects for software development:
• Place a brief explanatory comment at the start of every program
• Decompose programs into functions
• Be ruthless about eliminating duplication
• Always search for well-maintained software libraries that do what you need
• Test libraries before relying on them
• Give functions and variables meaningful names
• Make dependencies and requirements explicit
• Do not comment and uncomment sections of code to control a program's behavior
• Provide a simple example or test data set
• Submit code to a reputable DOI-issuing repository
5
https://arxiv.org/abs/1609.00037
6.
Best Practices for Scientific Computing
• This paper focuses on the development of scientific software for science
domains:
• Write programs for people, not computers
• Let the computer do the work
• Make incremental changes
• Do not repeat yourself (or others)
• Plan for mistakes
• Optimize software only after it works correctly
• Document design and purpose, not mechanics
• Collaborate
6
https://doi.org/10.1371/journal.pbio.1001745
7.
Scientific Software Best Practices
• This page highlights best-practices for scientific software development:
• Designing your software
• Software Requirements Specification
• Test-driven Development
• Version Control
• Branching Strategy
• Versioning of Releases
• Creating Citable Code
• Coding Guidelines and Code Review
• Continuous Integration
• Gamification
• Integrated Development Environment
7
https://scientific-software-best-practices.readthedocs.io/en/latest/
8.
Best Practices for Scientific Software
• This blog post focuses on domain science software:
• Hosting
• Packaging / installation
• Documentation
• Assistance
• Testing
• Academic publishing
8
https://software.ac.uk/blog/2017-11-29-best-practices-scientific-software
9.
However, there are no actual
recommendations for developing
scientific software frameworks!
Here it is our approach for the
WRENCH Software Framework…
9
10.
The WRENCH Simulation Framework
10
• Objective: Make it easy to develop simulators of complex
Cyberinfrastructure application executions
• Provides high-level, reusable simulation abstractions
• Produces accurate and scalable simulations
https://wrench-project.org
11.
Hosting: Open Source Project
• Some numbers
• > 26K lines of code
• C++, Javascript
• > 300 files
• ~3000 commits
• 16 contributors
• 12 releases
• 7 branches
• Version control
• Issues Tracking
11
https://github.com/wrench-project/wrench
19.
Gamification
• Getting a little badge to appear notifying you, gives a
(small) feeling of accomplishment and can be a good
motivator to write better pull requests
19
23.
Some very successful scientific
software framework examples…
23
24.
24
Pegasus Workflow
Management System
http://pegasus.isi.edu
• Open source project since 2001
• Documentation with examples
and tutorial
• Tests and continuous
Integration
• Thousands of citations
25.
Scikit-learn
• Open source Python
library used for
millions of data
scientists
• Tests code coverage:
98%
25
What is the best way to write secure and reliable applications?
- Good Enough Practices in Scientific Computing
(https://arxiv.org/abs/1609.00037)
- Best Practices for Scientific Computing (https://doi.org/10.1371/journal.pbio.1001745)
- Scientific Software Best Practices (https://scientific-software-best-practices.readthedocs.io/en/latest/)
- Best Practices for Scientific Software (https://software.ac.uk/blog/2017-11-29-best-practices-scientific-software)
Transcript
1.
Good Practices for Developing
Scientific Software Frameworks
The WRENCH framework example (and some others)
Rafael Ferreira da Silva
https://rafaelsilva.com
https://wrench-project.org
In collaboration with:
2.
What is the best way to write
secure and reliable applications?
2
5.
Good Enough Practices in Scientific Computing
• Among several recommendations, this paper emphasizes the following
aspects for software development:
• Place a brief explanatory comment at the start of every program
• Decompose programs into functions
• Be ruthless about eliminating duplication
• Always search for well-maintained software libraries that do what you need
• Test libraries before relying on them
• Give functions and variables meaningful names
• Make dependencies and requirements explicit
• Do not comment and uncomment sections of code to control a program's behavior
• Provide a simple example or test data set
• Submit code to a reputable DOI-issuing repository
5
https://arxiv.org/abs/1609.00037
6.
Best Practices for Scientific Computing
• This paper focuses on the development of scientific software for science
domains:
• Write programs for people, not computers
• Let the computer do the work
• Make incremental changes
• Do not repeat yourself (or others)
• Plan for mistakes
• Optimize software only after it works correctly
• Document design and purpose, not mechanics
• Collaborate
6
https://doi.org/10.1371/journal.pbio.1001745
7.
Scientific Software Best Practices
• This page highlights best-practices for scientific software development:
• Designing your software
• Software Requirements Specification
• Test-driven Development
• Version Control
• Branching Strategy
• Versioning of Releases
• Creating Citable Code
• Coding Guidelines and Code Review
• Continuous Integration
• Gamification
• Integrated Development Environment
7
https://scientific-software-best-practices.readthedocs.io/en/latest/
8.
Best Practices for Scientific Software
• This blog post focuses on domain science software:
• Hosting
• Packaging / installation
• Documentation
• Assistance
• Testing
• Academic publishing
8
https://software.ac.uk/blog/2017-11-29-best-practices-scientific-software
9.
However, there are no actual
recommendations for developing
scientific software frameworks!
Here it is our approach for the
WRENCH Software Framework…
9
10.
The WRENCH Simulation Framework
10
• Objective: Make it easy to develop simulators of complex
Cyberinfrastructure application executions
• Provides high-level, reusable simulation abstractions
• Produces accurate and scalable simulations
https://wrench-project.org
11.
Hosting: Open Source Project
• Some numbers
• > 26K lines of code
• C++, Javascript
• > 300 files
• ~3000 commits
• 16 contributors
• 12 releases
• 7 branches
• Version control
• Issues Tracking
11
https://github.com/wrench-project/wrench
19.
Gamification
• Getting a little badge to appear notifying you, gives a
(small) feeling of accomplishment and can be a good
motivator to write better pull requests
19
23.
Some very successful scientific
software framework examples…
23
24.
24
Pegasus Workflow
Management System
http://pegasus.isi.edu
• Open source project since 2001
• Documentation with examples
and tutorial
• Tests and continuous
Integration
• Thousands of citations
25.
Scikit-learn
• Open source Python
library used for
millions of data
scientists
• Tests code coverage:
98%
25