Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Cuashi2008revisited

291 views

Published on

The ideology behind the hydrological modelling I do. It is a revisiting of part of a talk I gave at CUAHSI biennial meeting in Boulder (CO) on July 2008. It promotes the modeling-by-components paradigm

Published in: Education
  • Be the first to comment

  • Be the first to like this

Cuashi2008revisited

  1. 1. Hydrological Modelling in 2016 Concepts and deployment hints Rigon R. Arpae, Parma, 17 Maggio 2017 MarzioTamer
  2. 2. !2 R. Rigon Introduction MODELS MademoiselleRose,ca.1820.EugèneDelacroix.
  3. 3. !3 DataParametersEquations Mass, momentum and energy conservation. Chemical transformations Forcings and observables Equation’s constant. In time! In space they are usually heterogeneous Models we are talking about are computer applications In the past they were built as monolithic programs R. Rigon Which kind of models
  4. 4. !4 I - Once a model, design and implemented as a monolithic software entity, has been deployed, its evolution is totally in the hands of the original developers. While this is a good thing for intellectual property rights and in a commercial environment, this is absolutely a bad thing for science and the way it is supposed to progress. RobbedfromaCCApresentation R. Rigon The old way
  5. 5. !5 II - Independent revisions and third-party contributions are nearly impossible and especially when the code is not available. Models falsification (in Popper sense) is usually impossible by other scientists than the original authors. III- Thus, model inter-comparison projects give usually unsatisfying results. Once complex models do not reproduce data it is usually very difficult to determine which process or parameterization was incorrectly implemented. R. Rigon The old way
  6. 6. !6 Q: HOW CAN WE BE MORE “GALILEIAN” ? A:YES, PRODUCING AND PROMOTING OPEN SOURCE MODELS. THIS HOWEVER IS NOT ENOUGH SINCE MODELS SHOULD BE STRUCTURALLY EASY TO UNDERSTAND, DOCUMENT, MODIFY, MAINTAIN,AND FAVOR PROCESSES ANALYSIS. R. Rigon The new way
  7. 7. !7 MODELLING, FOR WHO ? Which end user do you have in mind ? Baboon,PapiusAnubis R. Rigon No models for everyone
  8. 8. !8 • Prime users: take or prepare decisions at a political level • Technical users: prepare projects or maps for the primary users • Other end-users: national agencies, representative groups, etc.They may take or prepare decisions at national or regional level, or represent stakeholder groups. • Model and application developers/modellers: build models and targeted applications ModifiedfromRizzolietal.,2005 R. Rigon Users/Actors
  9. 9. !9 • Coders: implement models, applications and tools. • Linkers: link existing models and applications. • Runners: execute existing models, but they create and define scenarios. • Players: play simulations and experiments comparing scenarios and making analyses. • Viewers: view the players’ results, have a low level of interaction with the framework. • Providers: provide inputs and data to all other user roles. Roles ModifiedfromRizzolietal.,2005 R. Rigon Roles
  10. 10. !10 Modified from Rizzoli et al., ,2005 Roles Users Hard Coders Soft Coders Linkers Runners Player Viewers Providers Prime Other End Users Technical Researchers R. Rigon Users/Roles
  11. 11. !11 Thus there exists a “zoo” of users, each one with different abilities. Bundling data, algorithms and the graphical user interface of a model in an application with the old techniques makes the models very hard to re-use out of its original context. R. Rigon Reuse
  12. 12. !12 SO WHAT ? R. Rigon Solutions
  13. 13. !13 Object-oriented software development. O-O programming is nothing new, but it has proven to be a successful key to the design and implementation of modelling frameworks. Models and data can be seen as objects and therefore they can exploit properties such as encapsulation, polymorphism, data abstraction and inheritance. NEW (well relatively) MODELING PARADIGMS ModifiedfromRizzolietal.,2005 R. Rigon Software Engineering Solutions
  14. 14. !14 Component-oriented software development. Objects (models and data) should be packaged in components, exposing for re-use only their most important functions. Libraries of components can then be re-used and efficiently integrated across modelling frameworks.Yet, a certain degree of dependency of the model component from the framework can actually hinder reuse. NEW (well relatively) MODELING PARADIGMS ModifiedfromRizzolietal.,2005 R. Rigon Software Engineering Solutions
  15. 15. !15 R. Rigon Components
  16. 16. !16 Discrete units of software which are re-usable even outside the framework, both for model components and for tools components. Seamless and transparent access to data, which are made independent of the database layer. A number of tools (simulation, calibration, etc.) that the modeller will be free to use (including a visual modelling environment). A model repository to store your model (and simulations) and to share it with others. BENEFITS R. Rigon Benefits
  17. 17. !17 Tools for studying feedbacks among different processes. BENEFITS FOR SCIENTISTS Encapsulation of single processes or submodels MUCH MORE in the field of possibilities New educational tools and a “storage” of hydrological knowledge using appropriate onthologies R. Rigon Benefits
  18. 18. !18 BENEFITS FOR PROJECTS’ MANAGEMENT Easily tracking of software authorship. Developers concentrate on components and not on the overall Components are maintained. Not models.This can make easier to update modelling solutions Components are updated. Not models. Components are debugged and tested. Not models. Encapsulation helps. Well, model solutions integrity must also be preserved. R. Rigon Benefits
  19. 19. !19 T H E R E E X I S T S U C H M O D E L I N G INFRASTRUCTURE ? Economic modelling frameworks^. GAMS (general algebraic modelling system, http://www.gams.com) and GTAP (global trade analysis program, http:// www.gtap.agecon.purdue.edu ) are some of the most used modelling systems in the agro-economic domain.They can also account for social variables, such as unemployment. ^from Rizzoli et al., (Modeling Framework (SeamFrame) Requirements 2005 R. Rigon Existing Examples: not updated but useful
  20. 20. !20 T H E R E E X I S T S U C H M O D E L I N G INFRASTRUCTURE ? Environmental modelling frameworks. If we limit to the agricultural domain, the list is quite limited.There is no ‘real’ framework according to the definition, but APSIM, STICS and CropSyst provide some of the functionalities. In this area SEAMFRAME is an emerging technology.When we consider the water management sector, we find many examples, such as TIME (the invisible modelling environment), IMT, OpenMI, and OMS, and, to a certain respect, JUPITER-API. ^ extended from Rizzoli et al., (Modeling Framework (SeamFrame) Requirements 2005 R. Rigon Existing Examples: not updated but useful
  21. 21. !21 T H E R E E X I S T S U C H M O D E L I N G INFRASTRUCTURE ? Other modelling software environments of notable interest are SME, MMS, ICMS, Tarsier, Modcom, Simile, but they are integrated modelling environments, not frameworks.This means that they can be used to perform assessments, analyses, decision support, but they do not provide programming structures such as classes, components, objects, design patterns to be used to create end-user applications. ^from Rizzoli et al., Modeling Framework (SeamFrame) Requirements, 2005 R. Rigon Existing Examples: not updated but useful
  22. 22. !22 T H E R E E X I S T S U C H M O D E L I N G INFRASTRUCTURE ? Atmospheric Sciences: Earth Sciences Modeling Framework (ESMF) (including Earth System Curator) High Performance Computing: Common Component Architecture (CCA) R. Rigon Existing Examples: not updated but useful
  23. 23. !23 A F T E R 1 0 Y E A R S , W H Y T H E S E I N F R A S T RU C T U R E S D I D N OT EMERGE ? R. Rigon TOO INVASIVE ! TOO MANY COMPUTER SCIENTISTS,TOO FEW HYDROLOGISTS ? Existing Examples: here they are useful
  24. 24. !24 A BLUEPRINT ? Escher-Drawingshands,1948 R. Rigon Infrastructure design
  25. 25. !25 DataParametersEquations Mass, momentum and energy conservation. Chemical transformations Forcings and observables Equation’s constant. In time! In space they are heteorgeneous Numerics, boundary and initial conditions Data Assimilation. Data Models. Tools for Analysis. Calibration, derivation from proxies To Sum up R. Rigon Infrastructure design
  26. 26. !26 Decision making EVALUATION OF STRATEGIES THROUGH MODELS STRATEGIES FOR POLICY MAKERSDATA INTERPRETATION EVALUATION OF STRATEGIES THROUGH MODELSEVALUATION OF STRATEGIES THROUGH MODELS DATA INTERPRETATION DATA INTERPRETATION STRATEGIES FOR POLICY MAKERS STRATEGIES FOR POLICY MAKERS R. Rigon Infrastructure design
  27. 27. !27 PREREQUISITES General Programming LANGUAGE NEUTRAL PLATFORM NEUTRAL: Windows, Linux and Mac OPEN SOURCE TARGETED AT PERSONAL PRODUCTIVITY OF DIFFERENT USERS People come before program efficiency. BUSINNES NEUTRAL: GPL would be fine if encapsulated in components R. Rigon Infrastructure design
  28. 28. !28 PREREQUISITES Technologies ALLOWS WRAPPING OF EXISTING CODES BUT PROMOTES BETTER PROGRAMMING STRATEGIES DATA BASE AWARE DEPLOYABLE THROUGH THE WEB or as a web-server USES MULTICORES COMPLIANT OF STANDARDS (OGC, CUAHSI, OTHERS) R. Rigon Infrastructure design
  29. 29. !29 PREREQUISITES Documentation/Replicability WITH TOOLS THAT HELPS DOCUMENTATION COMPLIANT TO STANDARDS FOR DEFINING VARIABLES (e.g.VARIABLES AND PARAMETERS) MANAGED IN A PUBLIC REVISION CONTROL SYSTEM (e.g. GIT) HAVING A STANDARD WAY AND PLACES TO EXPOSE DOCUMENTATION R. Rigon Infrastructure design
  30. 30. !30 JAVA OMS GEOTools JGrassTools C/C++ Languages and infrastructures/libraries Rigon et al.
  31. 31. !31 JAVA OMS GEOTools JGrassTools C/C++ Python FORTRAN ESMF A competing solution Rigon et al.
  32. 32. !32 JAVA OMS GEOTools JGrassTools C/C++ OPENMI C# Another competing solution Rigon et al.
  33. 33. !33 tion. As with any EMF, fully embracing the OMS3 architecture requires a commitment to a structured model development process which may include the use of a version control system for model source code management or databases to store audit trails. Such features are important for institutionalized adoption of OMS3 but less critical for adherence by a single modeler. techniques such as parameterized types, higher level data struc- tures and/or object composition. The use of object-oriented design principles for modeling can be productive for a specific modeling project that has limited need for external reuse and extensibility. Extensive use of object-oriented design principles can be difficult for scientists to adopt in that adoption often entails a steep learning Fig. 1. OMS3 principle framework architecture. Please cite this article in press as: David, O., et al., A software engineering perspective on environmental modeling framework design: The Object Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design: The Object Modeling System. Environmental Modelling and Software, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006 OMS Rigon et al.
  34. 34. !34 deling components nction and continue ve characteristics of a modeling object interfaces to imple- methods to override, es to use. OMS3 uses specify and describe s and class methods ode quality of using rsus traditional API tudy comparing the hydrology models pplied several code ics of the different non-invasive frame- ncise model imple- ode and lower code ment. For example, ite model required quired between 450 mplementations had e modeling results. S model in OMS3 exchange items), and management of various execution states within components including “Initialize/Run/Finalize” as described by Peckham (2008). While object-oriented methods focus on abstraction, encapsula- tion, and localization of data and methods, their use can also lead to simulation systems where objects are highly co-dependent. To Fig. 2. OMS3 component architecture including data flow, execution phases, and encapsulation. A software engineering perspective on environmental modeling framework design: The Object Software (2012), doi:10.1016/j.envsoft.2012.03.006 David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design: The Object Modeling System. Environmental Modelling and Software, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006 Components again Rigon et al.
  35. 35. !35 (Duriancik et al., 2008). RUSLE2 has historically been used as a WindowsÔ-based desktop application to guide conservation planning and inventory erosion rates over large areas. The model provides a reusable computational engine that can be used without a user interface for model runs in other applications. RUSLE2’s water supply forecasts with sh of distributed-parameter, phy an Ensemble Streamflow P primary ESP model base (Leave and the PRMS hydrological wa be used to address a wide var information on the volume an improve water supply forecas ology is a modified version of National Weather Service (D synthesized meteorological da timeseries data used as model A visualization tool runn visual display of user-selec performs a frequency analys the simulated hydrograph tr historic years used with thei ance. Different options are analysis. One assumes that all an equal likelihood of occ weighting user-defined perio a priori information, are also b and Pacific Decadal Oscillatio fied in the ESP procedure, and separately for analysis. The P will provide timely foreca community in the western a major source of water supp Another modeling applicati the OMS3 framework is the c Ecosystem-Watershed) modeFig. 4. Cloud Services Innovation Platform (CSIP) software architecture. Please cite this article in press as: David, O., et al., A software engineering perspective on environmental mo Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006 David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design: The Object Modeling System. Environmental Modelling and Software, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006 CSIP Rigon et al.
  36. 36. !36 ulation of water quantity and quality in large watersheds ough et al., 2010). AgES-W consists of Java-based simulation modeling frameworks are currently under development wor with the primary purpose of integrating existing and futur Fig. 5. CSIP/OMS3-based mobile RUSLE2 erosion model application. O. David et al. / Environmental Modelling & Software xxx (2012) 1e13 David, O., Ascough, J. C., II, Lloyd, W., Green, T. R., Rojas, K. W., Leavesley, G. H., & Ahuja, L. R. (2012). A software engineering perspective on environmental modeling framework design: The Object Modeling System. Environmental Modelling and Software, 1–13. http://doi.org/10.1016/j.envsoft.2012.03.006 CSIP Rigon et al.
  37. 37. !37 Other companions Rigon et al.
  38. 38. !38 http://geoframe.blogspot.com Rigon et al.
  39. 39. !39 INTEGRATION One very diffuse platform in the operational environments is Deltares’ Fews. It is, unfortunately, not open source*: but free. However, it seems, that the chain of tools that derives from GEOframe can be seamlessly integrated in it. This is certainly a perspective to explore further Rigon et al. https://oss.deltares.nl/web/delft-fews/ *It is not available for MAC too
  40. 40. !40 Find this presentation at http://abouthydrology.blogspot.com Ulrici,2000? Other material at Questions ? R. Rigon http://abouthydrology.blogspot.it/2016/05/geoframe-system-for-doing-hydrology-by.html

×