This document summarizes Dr. Vahid Garousi's presentation on software engineering of scientific software. It discusses an experience developing OptimalPipeline, a software to optimize pumping costs in oil pipelines. A multi-disciplinary team applied software engineering practices to develop the tool over several iterations. Lessons learned included challenges with terminology differences and ensuring proper integration and testing. The presentation also reviewed the large worldwide community and literature on scientific software engineering. Possible areas for collaboration were discussed.

1. 1Dr. Vahid Garousi
Software engineering of scientific and engineering
software:
a few project experiences and a review of literature
Dr. Vahid Garousi
Associate Professor of Software Engineering
Information Technology Group
Wageningen University, Netherlands
www.vgarousi.com
www.wur.nl/inf
@vgarousi
Talk @ VORtech BV
Jan. 15, 2018

2. 2Dr. Vahid Garousi
About me
 An international background...
 Work experience:
 Wageningen University, the Netherlands, 2017-
 Hacettepe University, Ankara, Turkey, 2015-2017
 University of Calgary, Calgary, Canada, 2006-2014
 Founder, Maral Software Engineering Consulting Corporation, 2008-2014
 Education:
 PhD, Carleton University, Ottawa, Canada, 2006
 MSc, University of Waterloo, Waterloo, Canada, 2003
 BSc, Sharif University of Technology, Tehran, Iran, 2000
 Research expertise:
 Software Engineering, Software Testing and Quality Assurance (QA)
 Engineering (development) of scientific software
 Agile methodologies
 “Action research”: involvement in 40+ industry-academia collaborative projects in
several countries so far

3. 3Dr. Vahid Garousi
Wageningen University’s Information Technology Group (INF)
 12 staff members
 Collaborations with many academic and industry partners in the NL and
beyond

4. 4Dr. Vahid Garousi
Context
 Software is extensively used in all scientific disciplines. scientific software
 Scientific software in this context is broadly defined as software used for
scientific purposes, or software emerging from scientific research.
 Validity of research findings in science are critically dependent on quality of
software generating those findings

5. 5Dr. Vahid Garousi
Context
 But when developing scientific software...
 Often, Software Engineering principles and “best practices” (design patterns,
systematic code reuse, systematic testing, API design, etc.) are not applied
 Thus, there is an opportunity to build “better” scientific software with less
effort
Software
Engineers
/ Computer
Scientists
“Other”
(conventional)
Scientists
Opportunity to
build higher-
quality scientific
software with less
effort
Golden
triangle

6. 6Dr. Vahid Garousi
Outline of the talk
 Context
 Experience from engineering of a scientific software for
oil pipelines
 A brief review of:
 the world-wide community in this area
 the research literature in this area
 Possible areas of collaboration among us, and
Interactive discussions

7. 7Dr. Vahid Garousi
Engineering (developing) of a scientific software for oil pipelines
 Context: high cost ($/€) of pumping oil products across oil pipelines
 One pumping station per 50-300 KM
 A world-wide total of several billion dollars/year

8. 8Dr. Vahid Garousi
A software for reducing pumping costs in oil pipelines
 Pump stations consume either electric energy or fossil fuels, e.g.,
natural gas
 Everything is controlled / monitored by SCADA systems from a central
control center
 SCADA: Supervisory Control And Data Acquisition

9. 9Dr. Vahid Garousi
Engineering (developing) of a scientific software for oil pipelines
 A project proposal was submitted and it won a major funding provided by the
Canadian province of Alberta
 Industry partners: Enbridge and Pembina
 (Enbridge is the “largest” oil pipeline operator in the world)
 Project:
 Title: Engineering Intelligent Software Systems for Improving the Operational Efficiency of Oil Pipeline
Networks
 Timeline: 2007-2012
 Budget: $300,000 CND
 Several students were hired:

10. 10Dr. Vahid Garousi
Engineering (developing) of a scientific software for oil pipelines
 Team:
 Civil engineers (liquid hydraulics)
 Mechanical and chemical engineers (pipeline expertise)
 Software engineers (high-quality software development and
integration with existing systems) – team leadership
 Optimization experts (to get advice on building the optimization
model)
 Truly “multi-disciplinary”...
Mechanical and
chemical engineers:
Domain expertise
and case studies
Civil engineering and
optimization consulting
PI and
research team

11. 11Dr. Vahid Garousi
Software engineering started...
 It was an iterative process
 Everyone was involved in all phases

12. 12Dr. Vahid Garousi
The product: OptimalPipeline software
 In several iterations, we developed the product
 Functionality:
Output:
• Hourly speed of each pump
such that the total network-level
pumping costs are minimized
Input:
• Topology of your pipeline network
• Pipeline's mechanical characteristics
• Pumps' characteristics
• Multi-tariff Electricity power rates
• Contracted delivery volume of oil
product
A non-linear optimization
engine using genetic
algorithms (GA) was
developed

13. 13Dr. Vahid Garousi
The product: OptimalPipeline software
 Demo video: www.youtube.com/watch?v=VF8cf38wNOQ

14. 14Dr. Vahid Garousi
The product: OptimalPipeline software
 Screenshots:
 Outcome: a start-up company was formed and few licenses of the software
were sold.

15. 15Dr. Vahid Garousi
Software architecture of the software
 Integration with other systems in the companies
 We used Software Engineering best practices, e.g., the
Model–view–controller (MVC) architectural pattern
User
interacts
calls (uses)
calls (uses)
calls (uses)
To visualize the
pipeline network
The optiimzation
engine
To generate the
output charts
interacts with
SCADA systems
(Supervisory
Control And Data
Acquisition)

16. 16Dr. Vahid Garousi
Detailed architectural
design
 We used the Model–view–controller
(MVC) architectural pattern
View
Controller
Model
User

17. 17Dr. Vahid Garousi
Experience and lessons learned w.r.t. development of scientific software
 While the multi-disciplinary nature of the project made it quite successful, it
also caused challenges at times
 We observed success stories and challenges similar to what are reported in
the literature, e.g.:
 The vocabulary used by conventional scientists (engineers) and software
engineers are not the same! Reason: their background and expertise differ
 There is a need for both sides to learn from the other side
 With perseverance, the experience is fulfilling and the resulting outcome is
higher quality, if the system was to be developed ONLY by “one” of the two
sides
 Initially, conventional scientists (engineers) think that software engineering is
only “coding”, but later, they find that it is more than just coding, e.g., how to
systematically test? how to systematically design the system? etc.

18. 18Dr. Vahid Garousi
Experience and lessons learned w.r.t. development of scientific software
 Almost each phase had challenges and we learned lessons in it!
 How to ensure to capture all (most of) the requirements early on?
 How to ensure all the engineers (software, mechanical, civil, etc.) would
understand each other properly? The issue of “terminologies” and domain
expertise
 How to design to ensure proper integration with other systems (such as SCADA)
and also design for easy maintenance in future?
 How to write the code to ensure easy understandability and maintenance?
 How do we know the output are really optimal (lowest pumping energy)? How to
test the software systematically? And in each version (too much effort!)
 How can we maintain the software in the most cost-effective manner?

19. 19Dr. Vahid Garousi
More details

20. 20Dr. Vahid Garousi
My other involvement in development of scientific software
 Years: 1999-2002
 Collaborated with a team of Reservoir (Chemical) Engineers to develop a large Fortran
software (~120 KLOC) to simulate oil reservoirs (lots of modeling and simulation)

21. 21Dr. Vahid Garousi
Outline of the talk
 Context
 Experience from engineering of a scientific software for
oil pipelines
 A brief review of:
 the world-wide community in this area
 the research literature in this area
 Possible areas of collaboration among us, and
Interactive discussions

22. 22Dr. Vahid Garousi
The world-wide community in this area
 There is a very large community on engineering of scientific
software
 See some of the links online:

23. 23Dr. Vahid Garousi
The world-wide community
 In the NL...

24. 24Dr. Vahid Garousi
The world-wide community
 Many online videos from other groups active in this field around the
globe
US National Center for Supercomputing
Applications - NCSA

25. 25Dr. Vahid Garousi
The world-wide community

26. 26Dr. Vahid Garousi
There is also a vast “literature” in this area
 Many papers are published constantly in this area

27. 27Dr. Vahid Garousi
There is also a vast “literature” in this area
 We conducted a “survey” paper in 2012 on a pool of 130 papers in this
area

28. 28Dr. Vahid Garousi
There is also a vast “literature” in this area
 We maintain a pool of related works (literature): www.goo.gl/cdcnyN

29. 29Dr. Vahid Garousi
From our survey paper
 Pool: 130 papers in this area

30. 30Dr. Vahid Garousi
Some example advances: Efforts for testing scientific SW

31. 31Dr. Vahid Garousi
Some example advances: Efforts for testing scientific SW
 Metamorphic testing (MT): alleviates the test-oracle problem
by using some problem domain-specific properties, namely
metamorphic relationships (MRs) to verify the testing
outputs.
 The central idea: although it may be impossible to directly
test the correctness of any given test case, it may be
possible to verify the expected relationships of the outputs

32. 32Dr. Vahid Garousi
END of the talk
 Context
 Experience from engineering of a scientific software for
oil pipelines
 A brief review of:
 the world-wide community in this area
 the research literature in this area
 Possible areas of collaboration among us, and
Interactive discussions
Questions and discussions

33. 33Dr. Vahid Garousi
BACK-UP SLIDES

34. 34Dr. Vahid Garousi
Possible areas of collaboration among us
 We have a vision to build a
grass-roots community of
scientific software developers in
WUR
 What we in the Information
Technology Group can offer you:
 Development of scientific software
with high-quality and least effort,
which will be easy to maintain in
future
 Joint work and helping you with
software engineering aspects
 Result for you: Better software ->
Better research!
Let’s discuss possible
areas of collaboration
among us

35. 35Dr. Vahid Garousi
Outline of the talk
 Context
 What is “software engineering” after all? (in two slides!)
 Experience from engineering of a scientific software for
reducing pumping costs in oil pipelines
 A brief review of:
 the world-wide community in this area
 the research literature in this area
 Possible areas of collaboration among us, and
Interactive discussions

36. 36Dr. Vahid Garousi
Software engineering
 SE is “the application of a systematic, disciplined,
quantifiable approach to the design, development, testing,
operation, and maintenance of (large-scale) software
systems”
 History:

37. 37Dr. Vahid Garousi
Software engineering
Process:
Golden
triangle
In each phase, we keep asking:
how can we do it more
effectively (higher quality) and
efficiently (less costs)?

38. 38Dr. Vahid Garousi
Invitation to a survey by the NL-RSE
 https://softwaresaved.limequery.com/776594?lang=en