Cuashi2008revisited

Hydrological Modelling in 2016
Concepts and deployment hints
Rigon R.
Arpae, Parma, 17 Maggio 2017
MarzioTamer

!2
R. Rigon
Introduction
MODELS
MademoiselleRose,ca.1820.EugèneDelacroix.

!3
DataParametersEquations
Mass, momentum and
energy conservation.
Chemical
transformations
Forcings and
observables
Equation’s constant. In
time!
In space they are
usually heterogeneous
Models we are talking about are computer applications
In the past they were built as monolithic programs
R. Rigon
Which kind of models

!4
I - Once a model, design and implemented as a monolithic
software entity, has been deployed, its evolution is totally in the
hands of the original developers. While this is a good thing for
intellectual property rights and in a commercial environment, this
is absolutely a bad thing for science and the way it is
supposed to progress.
RobbedfromaCCApresentation
R. Rigon
The old way

!5
II - Independent revisions and third-party contributions are
nearly impossible and especially when the code is not available.
Models falsiﬁcation (in Popper sense) is usually impossible by
other scientists than the original authors.
III- Thus, model inter-comparison projects give usually unsatisfying
results. Once complex models do not reproduce data it is
usually very difﬁcult to determine which process or
parameterization was incorrectly implemented.
R. Rigon
The old way

!6
Q: HOW CAN WE BE MORE “GALILEIAN” ?
A:YES, PRODUCING AND PROMOTING OPEN
SOURCE MODELS. THIS HOWEVER IS NOT
ENOUGH SINCE MODELS SHOULD BE
STRUCTURALLY EASY TO UNDERSTAND,
DOCUMENT, MODIFY, MAINTAIN,AND FAVOR
PROCESSES ANALYSIS.
R. Rigon
The new way

!7
MODELLING, FOR WHO ?
Which end user do you have in mind ?
Baboon,PapiusAnubis
R. Rigon
No models for everyone

!8
• Prime users: take or prepare decisions at a
political level
• Technical users: prepare projects or maps for
the primary users
• Other end-users: national agencies,
representative groups, etc.They may take or
prepare decisions at national or regional level, or
represent stakeholder groups.
• Model and application developers/modellers:
build models and targeted applications
ModiﬁedfromRizzolietal.,2005
R. Rigon
Users/Actors

!9
• Coders: implement models, applications and
tools.
• Linkers: link existing models and applications.
• Runners: execute existing models, but they
create and deﬁne scenarios.
• Players: play simulations and experiments
comparing scenarios and making analyses.
• Viewers: view the players’ results, have a low
level of interaction with the framework.
• Providers: provide inputs and data to all other
user roles.
Roles
R. Rigon
Roles

!10
Modiﬁed from Rizzoli et al., ,2005
Roles
Users
Hard
Coders
Soft
Coders
Linkers Runners Player Viewers Providers
Prime
Other End
Users
Technical
Researchers
R. Rigon
Users/Roles

!11
Thus there exists a “zoo” of users, each one with different abilities.
Bundling data, algorithms and the graphical user interface of a
model in an application with the old techniques makes the models
very hard to re-use out of its original context.
R. Rigon
Reuse

!12
SO WHAT ?
R. Rigon
Solutions

!13
Object-oriented software development. O-O programming is
nothing new, but it has proven to be a successful key to the design
and implementation of modelling frameworks.
Models and data can be seen as objects and therefore they can
exploit properties such as encapsulation, polymorphism, data
abstraction and inheritance.
NEW (well relatively) MODELING PARADIGMS
R. Rigon
Software Engineering Solutions

!14
Component-oriented software development. Objects (models
and data) should be packaged in components, exposing for re-use
only their most important functions. Libraries of components can
then be re-used and efﬁciently integrated across modelling
frameworks.Yet, a certain degree of dependency of the model
component from the framework can actually hinder reuse.
NEW (well relatively) MODELING PARADIGMS
R. Rigon
Software Engineering Solutions

!16
Discrete units of software which are re-usable even outside the
framework, both for model components and for tools components.
Seamless and transparent access to data, which are
made independent of the database layer.
A number of tools (simulation, calibration, etc.) that the
modeller will be free to use (including a visual modelling
environment).
A model repository to store your model (and simulations)
and to share it with others.
BENEFITS
R. Rigon
Beneﬁts

!17
Tools for studying feedbacks among different processes.
BENEFITS FOR SCIENTISTS
Encapsulation of single processes or submodels
MUCH MORE in the ﬁeld of possibilities
New educational tools and a “storage” of hydrological
knowledge using appropriate onthologies
R. Rigon
Beneﬁts

!18
BENEFITS FOR PROJECTS’ MANAGEMENT
Easily tracking of software authorship. Developers
concentrate on components and not on the overall
Components are maintained. Not models.This can make
easier to update modelling solutions
Components are updated. Not models.
Components are debugged and tested. Not models.
Encapsulation helps.
Well, model solutions integrity must also be preserved.
R. Rigon
Beneﬁts

!19
T H E R E E X I S T S U C H M O D E L I N G
INFRASTRUCTURE ?
Economic modelling frameworks^. GAMS (general
algebraic modelling system, http://www.gams.com) and GTAP
(global trade analysis program, http://
www.gtap.agecon.purdue.edu ) are some of the most used
modelling systems in the agro-economic domain.They can also
account for social variables, such as unemployment.
^from Rizzoli et al., (Modeling Framework (SeamFrame)
Requirements 2005
R. Rigon
Existing Examples: not updated but useful

!20
INFRASTRUCTURE ?
Environmental modelling frameworks. If we limit to the
agricultural domain, the list is quite limited.There is no ‘real’
framework according to the deﬁnition, but APSIM, STICS
and CropSyst provide some of the functionalities. In this area
SEAMFRAME is an emerging technology.When we consider
the water management sector, we ﬁnd many examples, such
as TIME (the invisible modelling environment), IMT, OpenMI,
and OMS, and, to a certain respect, JUPITER-API.
^ extended from Rizzoli et al., (Modeling Framework
(SeamFrame) Requirements 2005
R. Rigon

!21
INFRASTRUCTURE ?
Other modelling software environments of notable
interest are SME, MMS, ICMS, Tarsier, Modcom,
Simile, but they are integrated modelling environments, not
frameworks.This means that they can be used to perform
assessments, analyses, decision support, but they do not provide
programming structures such as classes, components, objects,
design patterns to be used to create end-user applications.
^from Rizzoli et al., Modeling Framework (SeamFrame)
Requirements, 2005
R. Rigon

!22
INFRASTRUCTURE ?
Atmospheric Sciences: Earth Sciences Modeling Framework
(ESMF) (including Earth System Curator)
High Performance Computing: Common Component
Architecture (CCA)
R. Rigon

!23
A F T E R 1 0 Y E A R S , W H Y T H E S E
I N F R A S T RU C T U R E S D I D N OT
EMERGE ?
R. Rigon
TOO INVASIVE !
TOO MANY COMPUTER
SCIENTISTS,TOO FEW
HYDROLOGISTS ?
Existing Examples: here they are useful

!24
A BLUEPRINT ?
Escher-Drawingshands,1948
R. Rigon
Infrastructure design

!25
DataParametersEquations
Mass, momentum
and energy
conservation.
Chemical
transformations
Forcings and
observables
Equation’s
constant. In time!
In space they are
heteorgeneous
Numerics,
boundary and
initial conditions
Data Assimilation.
Data Models. Tools
for Analysis.
Calibration,
derivation from
proxies
To Sum up
R. Rigon

!26
Decision making
EVALUATION OF STRATEGIES THROUGH
MODELS
STRATEGIES FOR POLICY MAKERSDATA INTERPRETATION
EVALUATION OF STRATEGIES THROUGH
MODELSEVALUATION OF STRATEGIES THROUGH
MODELS
DATA INTERPRETATION
DATA INTERPRETATION
STRATEGIES FOR POLICY MAKERS
STRATEGIES FOR POLICY MAKERS
R. Rigon

!27
PREREQUISITES General
Programming LANGUAGE NEUTRAL
PLATFORM NEUTRAL: Windows, Linux and Mac
OPEN SOURCE
TARGETED AT PERSONAL PRODUCTIVITY OF
DIFFERENT USERS
People come before program efﬁciency.
BUSINNES NEUTRAL: GPL would be ﬁne if encapsulated in
components
R. Rigon

!28
PREREQUISITES Technologies
ALLOWS WRAPPING OF EXISTING CODES BUT
PROMOTES BETTER PROGRAMMING STRATEGIES
DATA BASE AWARE
DEPLOYABLE THROUGH THE WEB or as a web-server
USES MULTICORES
COMPLIANT OF STANDARDS (OGC, CUAHSI,
OTHERS)
R. Rigon

!29
PREREQUISITES Documentation/Replicability
WITH TOOLS THAT HELPS DOCUMENTATION
COMPLIANT TO STANDARDS FOR DEFINING VARIABLES
(e.g.VARIABLES AND PARAMETERS)
MANAGED IN A PUBLIC REVISION CONTROL
SYSTEM (e.g. GIT)
HAVING A STANDARD WAY AND PLACES TO
EXPOSE DOCUMENTATION
R. Rigon

!30
JAVA
OMS
GEOTools
JGrassTools
C/C++
Languages and infrastructures/libraries
Rigon et al.

!31
JAVA
OMS
GEOTools
JGrassTools
C/C++
Python
FORTRAN
ESMF
A competing solution
Rigon et al.

!32
JAVA
OMS
GEOTools
JGrassTools
C/C++
OPENMI
C#
Another competing solution
Rigon et al.

!33
tion. As with any EMF, fully embracing the OMS3 architecture
requires a commitment to a structured model development process
which may include the use of a version control system for model
source code management or databases to store audit trails. Such
features are important for institutionalized adoption of OMS3 but
less critical for adherence by a single modeler.
techniques such as parameterized types, higher level data struc-
tures and/or object composition. The use of object-oriented design
principles for modeling can be productive for a speciﬁc modeling
project that has limited need for external reuse and extensibility.
Extensive use of object-oriented design principles can be difﬁcult
for scientists to adopt in that adoption often entails a steep learning
Fig. 1. OMS3 principle framework architecture.
Please cite this article in press as: David, O., et al., A software engineering perspective on environmental modeling framework design: The Object
Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design:
The Object Modeling System. Environmental Modelling and Software, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006
OMS
Rigon et al.

!34
deling components
nction and continue
ve characteristics of
a modeling object
interfaces to imple-
methods to override,
es to use. OMS3 uses
specify and describe
s and class methods
ode quality of using
rsus traditional API
tudy comparing the
hydrology models
pplied several code
ics of the different
non-invasive frame-
ncise model imple-
ode and lower code
ment. For example,
ite model required
quired between 450
mplementations had
e modeling results.
S model in OMS3
exchange items), and management of various execution states
within components including “Initialize/Run/Finalize” as described
by Peckham (2008).
While object-oriented methods focus on abstraction, encapsula-
tion, and localization of data and methods, their use can also lead to
simulation systems where objects are highly co-dependent. To
Fig. 2. OMS3 component architecture including data ﬂow, execution phases, and
encapsulation.
A software engineering perspective on environmental modeling framework design: The Object
Software (2012), doi:10.1016/j.envsoft.2012.03.006
David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design:
Components again
Rigon et al.

!35
(Duriancik et al., 2008). RUSLE2 has historically been used as
a WindowsÔ-based desktop application to guide conservation
planning and inventory erosion rates over large areas. The model
provides a reusable computational engine that can be used without
a user interface for model runs in other applications. RUSLE2’s
water supply forecasts with sh
of distributed-parameter, phy
an Ensemble Streamflow P
primary ESP model base (Leave
and the PRMS hydrological wa
be used to address a wide var
information on the volume an
improve water supply forecas
ology is a modified version of
National Weather Service (D
synthesized meteorological da
timeseries data used as model
A visualization tool runn
visual display of user-selec
performs a frequency analys
the simulated hydrograph tr
historic years used with thei
ance. Different options are
analysis. One assumes that all
an equal likelihood of occ
weighting user-defined perio
a priori information, are also b
and Pacific Decadal Oscillatio
fied in the ESP procedure, and
separately for analysis. The P
will provide timely foreca
community in the western
a major source of water supp
Another modeling applicati
the OMS3 framework is the c
Ecosystem-Watershed) modeFig. 4. Cloud Services Innovation Platform (CSIP) software architecture.
Please cite this article in press as: David, O., et al., A software engineering perspective on environmental mo
Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006
CSIP
Rigon et al.

!36
ulation of water quantity and quality in large watersheds
ough et al., 2010). AgES-W consists of Java-based simulation
modeling frameworks are currently under development wor
with the primary purpose of integrating existing and futur
Fig. 5. CSIP/OMS3-based mobile RUSLE2 erosion model application.
O. David et al. / Environmental Modelling & Software xxx (2012) 1e13
CSIP
Rigon et al.

!37
Other companions
Rigon et al.

!38
http://geoframe.blogspot.com
Rigon et al.

!39
INTEGRATION
One very diffuse platform in the operational environments
is Deltares’ Fews.
It is, unfortunately, not open source*: but free.
However, it seems, that the chain of tools that derives
from GEOframe can be seamlessly integrated in it.
This is certainly a perspective to explore further
Rigon et al.
https://oss.deltares.nl/web/delft-fews/
*It is not available for MAC too

!40
Find this presentation at
http://abouthydrology.blogspot.com
Ulrici,2000?
Other material at
Questions ?
R. Rigon
http://abouthydrology.blogspot.it/2016/05/geoframe-system-for-doing-hydrology-by.html

Cuashi2008revisited

More Related Content

Viewers also liked

Similar to Cuashi2008revisited

More from Riccardo Rigon

Recently uploaded

Cuashi2008revisited