Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Parma 2016-05-17 - JGrass-NewAGE - Some About The State of Art

234 views

Published on

This describes the motivation behind the JGrass-NewAGE infrastructure. It also shows the main components that were implemented. Finally it shows and comments some case studies and some use cases

Published in: Education
  • Be the first to comment

Parma 2016-05-17 - JGrass-NewAGE - Some About The State of Art

  1. 1. The JGrass-NewAGE system essentials Concepts Rigon R. Arpae, Parma, 17 Maggio 2017 GiuseppePenone
  2. 2. !2 R. Rigon Introduction MODELS MademoiselleRose,ca.1820.EugèneDelacroix.
  3. 3. !3 DataParametersEquations Mass, momentum and energy conservation. Chemical transformations Forcings and observables Equation’s constant. In time! In space they are usually heterogeneous Models we are talking about are computer applications In the past they were built as monolithic programs R. Rigon Which kind of models
  4. 4. !4 I - Once a model, design and implemented as a monolithic software entity, has been deployed, its evolution is totally in the hands of the original developers. While this is a good thing for intellectual property rights and in a commercial environment, this is absolutely a bad thing for science and the way it is supposed to progress. RobbedfromaCCApresentation R. Rigon The old way
  5. 5. !5 II - Independent revisions and third-party contributions are nearly impossible and especially when the code is not available. Models falsification (in Popper sense) is usually impossible by other scientists than the original authors. III- Thus, model inter-comparison projects give usually unsatisfying results. Once complex models do not reproduce data it is usually very difficult to determine which process or parameterization was incorrectly implemented. R. Rigon The old way
  6. 6. !6 Q: HOW CAN WE BE MORE “GALILEIAN” ? A:YES, PRODUCING AND PROMOTING OPEN SOURCE MODELS. THIS HOWEVER IS NOT ENOUGH SINCE MODELS SHOULD BE STRUCTURALLY EASY TO UNDERSTAND, DOCUMENT, MODIFY, MAINTAIN,AND FAVOR PROCESSES ANALYSIS. R. Rigon The new way
  7. 7. !7 MODELLING, FOR WHO ? Which end user do you have in mind ? Baboon,PapiusAnubis R. Rigon No models for everyone
  8. 8. !8 Modified from Rizzoli et al., ,2005 Roles Users Hard Coders Soft Coders Linkers Runners Player Viewers Providers Prime Other End Users Technical Researchers R. Rigon Users/Roles
  9. 9. !9 SO WHAT ? R. Rigon Solutions
  10. 10. !10 Component-oriented software development. Objects (models and data) should be packaged in components, exposing for re-use only their most important functions. Libraries of components can then be re-used and efficiently integrated across modelling frameworks.Yet, a certain degree of dependency of the model component from the framework can actually hinder reuse. NEW (well relatively) MODELING PARADIGMS ModifiedfromRizzolietal.,2005 R. Rigon Software Engineering Solutions
  11. 11. !11 R. Rigon Components
  12. 12. !12 A F T E R 1 0 Y E A R S , W H Y T H E S E SOFTWARES BY COMPONENTS I N F R A S T RU C T U R E S D I D N OT EMERGE ? R. Rigon Existing Examples ? TOO INVASIVE ! TOO MANY COMPUTER SCIENTISTS, TOO FEW HYDROLOGISTS ?
  13. 13. !13 A BLUEPRINT ? Escher-Drawingshands,1948 R. Rigon Infrastructure design
  14. 14. !14 DataParametersEquations Mass, momentum and energy conservation. Chemical transformations Forcings and observables Equation’s constant. In time! In space they are heteorgeneous Numerics, boundary and initial conditions Data Assimilation. Data Models. Tools for Analysis. Calibration, derivation from proxies To Sum up R. Rigon Infrastructure design
  15. 15. !15 Decision making EVALUATION OF STRATEGIES THROUGH MODELS STRATEGIES FOR POLICY MAKERSDATA INTERPRETATION EVALUATION OF STRATEGIES THROUGH MODELSEVALUATION OF STRATEGIES THROUGH MODELS DATA INTERPRETATION DATA INTERPRETATION STRATEGIES FOR POLICY MAKERS STRATEGIES FOR POLICY MAKERS R. Rigon Infrastructure design
  16. 16. !16 PREREQUISITES General Programming LANGUAGE NEUTRAL PLATFORM NEUTRAL: Windows, Linux and Mac OPEN SOURCE TARGETED AT PERSONAL PRODUCTIVITY OF DIFFERENT USERS People come before program efficiency. BUSINESS NEUTRAL: GPL would be fine if encapsulated in components R. Rigon Infrastructure design
  17. 17. !17 PREREQUISITES Technologies ALLOWS WRAPPING OF EXISTING CODES BUT PROMOTES BETTER PROGRAMMING STRATEGIES DATA BASE AWARE DEPLOYABLE THROUGH THE WEB or as a web-server USES MULTICORES COMPLIANT OF STANDARDS (OGC, CUAHSI, OTHERS) R. Rigon Infrastructure design
  18. 18. !18 PREREQUISITES Documentation/Replicability WITH TOOLS THAT HELPS DOCUMENTATION COMPLIANT TO STANDARDS FOR DEFINING VARIABLES (e.g.VARIABLES AND PARAMETERS) MANAGED IN A PUBLIC REVISION CONTROL SYSTEM (e.g. GIT) HAVING A STANDARD WAY AND PLACES TO EXPOSE DOCUMENTATION R. Rigon Infrastructure design
  19. 19. The JGrass-NewAGE system essentials Deployment choices Rigon R, Formetta G., Antonello A., Franceschi S. Arpae, Parma, 17 Maggio 2017 GiuseppePenone
  20. 20. !20 JAVA OMS GEOTools JGrassTools C/C++ Languages and infrastructures/libraries Rigon et al. Formetta et al., 2014
  21. 21. !21 JAVA OMS GEOTools JGrassTools C/C++ Python FORTRAN ESMF A competing solution Rigon et al. Formetta et al., 2014
  22. 22. !22 JAVA OMS GEOTools JGrassTools C/C++ OPENMI C# Another competing solution Rigon et al. Formetta et al., 2014
  23. 23. !23 tion. As with any EMF, fully embracing the OMS3 architecture requires a commitment to a structured model development process which may include the use of a version control system for model source code management or databases to store audit trails. Such features are important for institutionalized adoption of OMS3 but less critical for adherence by a single modeler. techniques such as parameterized types, higher level data struc- tures and/or object composition. The use of object-oriented design principles for modeling can be productive for a specific modeling project that has limited need for external reuse and extensibility. Extensive use of object-oriented design principles can be difficult for scientists to adopt in that adoption often entails a steep learning Fig. 1. OMS3 principle framework architecture. Please cite this article in press as: David, O., et al., A software engineering perspective on environmental modeling framework design: The Object Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006 OMS Rigon et al. David et al., 2012
  24. 24. !24 deling components nction and continue ve characteristics of a modeling object interfaces to imple- methods to override, es to use. OMS3 uses specify and describe s and class methods ode quality of using rsus traditional API tudy comparing the hydrology models pplied several code ics of the different non-invasive frame- ncise model imple- ode and lower code ment. For example, ite model required quired between 450 mplementations had e modeling results. S model in OMS3 exchange items), and management of various execution states within components including “Initialize/Run/Finalize” as described by Peckham (2008). While object-oriented methods focus on abstraction, encapsula- tion, and localization of data and methods, their use can also lead to simulation systems where objects are highly co-dependent. To Fig. 2. OMS3 component architecture including data flow, execution phases, and encapsulation. A software engineering perspective on environmental modeling framework design: The Object Software (2012), doi:10.1016/j.envsoft.2012.03.006 Components again Rigon et al. David et al., 2012
  25. 25. !25 (Duriancik et al., 2008). RUSLE2 has historically been used as a WindowsÔ-based desktop application to guide conservation planning and inventory erosion rates over large areas. The model provides a reusable computational engine that can be used without a user interface for model runs in other applications. RUSLE2’s water supply forecasts with sh of distributed-parameter, phy an Ensemble Streamflow P primary ESP model base (Leave and the PRMS hydrological wa be used to address a wide var information on the volume an improve water supply forecas ology is a modified version of National Weather Service (D synthesized meteorological da timeseries data used as model A visualization tool runn visual display of user-selec performs a frequency analys the simulated hydrograph tr historic years used with thei ance. Different options are analysis. One assumes that all an equal likelihood of occ weighting user-defined perio a priori information, are also b and Pacific Decadal Oscillatio fied in the ESP procedure, and separately for analysis. The P will provide timely foreca community in the western a major source of water supp Another modeling applicati the OMS3 framework is the c Ecosystem-Watershed) modeFig. 4. Cloud Services Innovation Platform (CSIP) software architecture. Please cite this article in press as: David, O., et al., A software engineering perspective on environmental mo Modeling System, Environmental Modelling & Software (2012), doi:10.1016/j.envsoft.2012.03.006 CSIP Rigon et al. David et al., 2012
  26. 26. !26 ulation of water quantity and quality in large watersheds ough et al., 2010). AgES-W consists of Java-based simulation modeling frameworks are currently under development wor with the primary purpose of integrating existing and futur Fig. 5. CSIP/OMS3-based mobile RUSLE2 erosion model application. O. David et al. / Environmental Modelling & Software xxx (2012) 1e13 CSIP Rigon et al. David et al., 2012
  27. 27. !27 Other companions Rigon et al.
  28. 28. !28 http://geoframe.blogspot.com Rigon et al.
  29. 29. The JGrass-NewAGE system essentials Hydrology Arpae, Parma, 17 Maggio 2017 GiuseppePenone Rigon R, Formetta G. Bancheri M., Serafin F., Abera W.
  30. 30. !30 Kriging • Ordinary Kriging and detrended kriging and their local versions: results are in form of raster maps or shapefiles for selected points Based on the in situ data, it selects the best variogram (VGM) model, without any human decision, and optimises VGM parameters automatically at each time steps. Selection ofVGM model is NOT efficient (so far). What is there Rigon et al. Formetta, 2013
  31. 31. !31 • Separate rain from snow based on temperature: results are in form of raster maps or shapefiles for selected points It can be used conjointly with calibrators and satellite (e.g. MODIS) data to obtain local estimates of the parameters. RainSnow What is there Rigon et al. Formetta et al. 2014
  32. 32. !32 • Implements degree-day, Casorzi-Dalla Fontana and Hocks methods: needs radiation components. Results are in form of raster maps or shapefiles for selected points Snow What is there Rigon et al. Formetta et al. 2014
  33. 33. !33 • Priestley Taylor, FAO and Penman-Monteith versions. Various strategies were adopted to calibrate parameters. Only PT has been throughly tested and applied. ET What is there Rigon et al. Formetta, 2013
  34. 34. !34 Adige • Implements Hymod and separation of basin area in sub- catchments numbered according to a modification of the Pfastetter algorithm. Probably next version needs to be split apart into two or three components. What is there Rigon et al. Formetta et al., 2011
  35. 35. !35 LWRB SWRB • Shortwave and longwave radiation estimation. Contains algorithms for estimating shadows according to the geometry of complex terrain. They also have parameterisation for cloud cover. What is there Rigon et al. Formetta et al., 2013 Formetta et al., 2016
  36. 36. !36 LUCA Particle Swarm • Calibration tools. The first implements classic shuffle- complex evolution tools. They are part of OMS core. What is there Rigon et al. David et al., 2012
  37. 37. !37 deSaintVenant • Integration of de Saint-Venant 1D equation (part of Jgrasstools) What is there Rigon et al. http://abouthydrology.blogspot.it/search/label/de%20Saint-Venant%20equation
  38. 38. !38 A - AGEs To be checked B- JGrass-NewAGE (https://github.com/geoframecomponents) [Adige] BP- Backward probabilities Clearness Index ET FP -Forward probabilities [Kriging] NetRadiation LWRB - RainSnow SWB (Simple Water Budget) SWRB Snow C - JGrassTools (http://moovida.github.io/jgrasstools/) More than 50 components An index Rigon et al.
  39. 39. !39 D - OMS (https://alm.engr.colostate.edu) LUCA Particle Swarm And the whole infrastructure for running them all An index Rigon et al.
  40. 40. The JGrass-NewAGE system essentials Case studies and Use cases Arpae, Parma, 17 Maggio 2017 GiuseppePenone Abera W., Formetta G., Bancheri M., Serafin F., Abera W., Rigon R.
  41. 41. !41 (4.1) @t = Jk(t)+ i Qki(t)° ETk(t)°Qk(t) for an appropriate set of elementary control volumes connected together. In Eq.(5.1), S [L3 ] represents the total water storage of the basin, J [L3 T°1 ], ET [L3 T°1 ], and Q [L3 T°1 ] are precipitation, evapotranspiration, and runoff (surface and groundwater) respectively. The Qis represent input fluxes, of the same nature of Q, coming from adjacent control volumes. a b Figure 4.1: The location of the Posina basin in the Northeast of Italy (a) and DEM elava- tion, location of rain gauges and hydrometer stations, subbasin-channel link partitions used for this modelling (b). It is clear that Eq.(5.1) is governed by two types of terms, which can be easily identi- fied as “inputs" and “outputs". The outputs are certainly evapotranspiration, ET, and discharges, Q, including the Qis, because they come from the assembly of control volumes. The inputs are J(t), but this term has to be split into rainfall and snowfall. Moreover, other inputs are ancillary to the estimation of outputs, in particular temperature, T and radiation Rn. Another input of the equation is the definition of the domain of integration and its“granularity", i.e. its partition into elements for which a singe value of the state variables is produced. In this paper we discuss the estimation of all of these input quantities, with the Posina A small (114 km2) basin in Vicenza province, flowing into the Brenta river Abera et al. A small basin Abera, 2016
  42. 42. !42 method; Isaaks et al., 1989), based on removing one data point at a time and performing the interpolation for the location of the removed point using the remaining meteo-stations. Finally, for this paper, kriging is used to generate time series of meterological forcings for the centroid of each HRU. These forcings, for the purposes of this paper, are kept constant over the whole HRU area. Figure 4.3: The Spatial interpolation component of the NewAge system (SI-NewAge). The figure shows how different components are connected together, here the variogram (semivariogram) component solves for the spatial structure of measured data in the form of an experimental variogram. The particle swarm optimization algorithm uses the experimental variogram to identify the best theoretical semivariogram and optimal parameter sets for each time step. Lastly, Kriging uses the best semivariogram model Calibration of Kriging parameters Abera et al. Schemes of work Abera, 2016
  43. 43. !43 value of Ωrank, the higher the correlation between Js and snow albedo. Those parameters producing the highest Ωrank are used to model the hourly time steps of snowfall for each HRU. The derivation of snow separation parameters for each HRU is possible, however, as is pertinent to the overall analysis of other components of the study, single, global and optimized values of Eq.(4.3) parameters are derived. Figure 4.4: The Snow separation component, outlining how the MODIS snow products are used to calibrate the spatial snow accumulation ( Eq. 4.3). The dashed line shows the iterative (calibration) process to optimize the equation. Due to the time step differences between MODIS and the separation model output, the manual calibration is preferred in this case. Calibration of snow-rainfall separation Abera et al. Schemes of work Abera, 2016
  44. 44. !44 basin outlet, but in this application we excluded it because at these scales (of around ten kilometers) travel time in channels is irrelevant (D’Odorico and Rigon, 2003). Eventually the Hymod component provides an estimate of the discharge at each link of the river network of the watershed, downstream to the HRUs. ADIGE Figure 5.2: The HYmod component of NewAge system and its input providing compo- nents. It shows how different components are connected, here kriging, SWE, ETP, and calibration component connected with Adige to solve the runoff at high spatial and temporal resolution. The detail discussion about each component can be referred at its respective section. Calibration of the overall system Abera et al. Schemes of work Abera, 2016
  45. 45. !45 CHAPTER 5. ESTIMATING WATER BUDGET MODELLING OUTPUTS AND STORAGE COMPONENT 0 1000 2000 3000 Prainfall Psnow Precipi,J(mm) 0 1000 2000 94/5 95/6 96/7 97/8 98/9 99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08 08/09 09/10 10/11 11/12 Q AET S Watercomponents,AET,S(mm) Hydrological years Figure 5.11: Water budget components of the basin and its annual variabilities from 1994/95 to 2011/2012. It shows the relative share (the size of the bars) of the three components (Q, ET and S) of the total available water J. Annual budget Abera et al. The idea is that JGrass-NewAGE obtain water budgets Aberaetal,inpreparation,2016b
  46. 46. !46 CHAPTER 5. ESTIMATING WATER BUDGET MODELLING OUTPUTS AND STORAGE COMPONENT This could have been deduced from the data alone, However, seeing it with the other budget components enlighten the complexity of the interactions actually in place. 0 100 200 300 400 500 01-2012 02-2012 03-2012 04-2012 05-2012 06-2012 07-2012 08-2012 09-2012 10-2012 11-2012 12-2012 Date(month) Q,ET,S(mm/month) Q ET S 0 100 200 300 J(mm/month) Figure 5.12: The same as figure 5.11, but monthly variability for the year 2012. Monthly budget (temporal) Abera et al. The idea is that JGrass-NewAGE obtain water budgets Aberaetal,inpreparation,2016b
  47. 47. !47 J 80 120 160 200 Q 40 80 160 ET 20 40 60 S JanAprJulOct −150 −100 −50 0 50 Figure 5.13: The spatial variability of the long term mean monthly water budget com- ponents (J, ET, Q, S). For reason of visibility, the color scale is for each component separately. Monthly budget (spatial) Abera et al. The idea is that JGrass-NewAGE obtain water budgets Aberaetal,inpreparation,2016b
  48. 48. !48 Events Abera et al. But events are equally likely well reproduced Aberaetal,inpreparation,2016b
  49. 49. !49 6.1. INTRODUCTION 10 20 30 40 50 Long Lat a 8 9 10 11 12 13 36 38 40 Long Lat 1000 2000 3000 4000 Elevation(m) Lat Station Lake Tana b Figure 6.1: The geographic location of Upper Blue Nile basin in the Nile basin (a) and digitale elevation model of the basin (b). The points in figure b are the meteorological stations used for this study. Several validation studies of SREs have been conducted in the Ethiopian UBN basin (Dinku et al., 2007, 2008; Haile et al., 2013; Gebremichael et al., 2014; Worqlul et al., 2014; Romilly and Gebremichael, 2011; Hirpa et al., 2010; Habib et al., 2012). For instance, two comparative studies by Dinku et al. (2007) and Dinku et al. (2008) on high Blue Nile (175000 Km2) Abera et al. Larger rivers Aberaetal,inpreparation,2016c
  50. 50. !50 CMORPH is better in estimating ground-gauge rainfall using the two previous statistics (i.e., r and RMSE), it is underestimating by 72%, thus being the most biased product of the five SREs. This could be because CMORPH is only based on satellite products, and not corrected using ground data as 3B42V7. TAMSAT, on average, is underestimating rainfall by 30%. CorrelationRMSEBIAS 3B42V7 CMORPH CFSR SM2R-CCI TAMSAT 8 9 10 11 12 13Lat Correlation <0.2 (0.2,0.3] (0.3,0.4] (0.4,0.5] (0.5,0.6] (0.6,0.7] 8 9 10 11 12 13 Lat RMSE(mm/day) [4, 6] (6, 8] (8, 10] (10, 12] (12, 14] >14 8 9 10 11 12 13 36 38 40 36 38 40 36 38 40 36 38 40 36 38 40 Long Lat BIAS (-0.9,-0.6] (-0.6,-0.3] (-0.3,-0.1] (-0.1,0.1] (0.1,0.3] (0.3,0.6] (0.6,1.4] Figure 6.4: The spatial distribution of GOF values for different SREs: correlation coeffi- cient (first row), RMSE (second row) and Bias (third row). The spatial distribution of the the three GOF values (r, RMSE, BIAS) are presented in figure 6.4. Overall the distribution of the statistics can depict a spatial pattern, i.e., the correlations in the eastern and northeastern part of the basin are higher than western and southwestern part. Similar pattern can be inferred from the RMSE and BIAS Satellites products comparison Abera et al. Approached with satellite data Aberaetal,2016a
  51. 51. !51 6.5. RESULTS AND DISCUSSIONS A.Mehal Meda B.Debre Markos C.Assosa 0 1000 2000 3000 0 100 200 300 0 100 200 300 0 100 200 300 SREs Gauge observations CFSR CMORPH SM2R-CCI TAMSAT 3B42V7 MeanCumulativerainfall(mm) Days of year Mehal_Meda Debre_Markos Assosa Figure 6.6: Annual mean cumulative rainfall estimations based on five SREs and gauges data. these two kinds of SREs (e.g., SM2R-CCI and CMORPH or 3B42V7 or TAMSAT). Among the five SREs, TAMSAT has the highest detection capacity for lowest rainfall intensities (91%). For all classes, TAMSAT has the highest missing rate and the highest recorded is for the 0.1-2 mm observed rainfall class (54%), while the systematic bias Big Bias Abera et al. Which are not always good Aberaetal,2016a
  52. 52. !52 function of basin water storage, for instance Q and ET, good estimation of water storage of a model has inference to its reasonable computation of other fluxes as well (Döll et al., 2014). GRACE data is an extraordinary resource to assess the over all performance of the simulation, at least at the basin scale. 8 9 10 11 12 35 36 37 38 39 40 long lat 3.0 3.5 4.0 4.5 5.0 Precip(mm/day) 8 9 10 11 12 35 36 37 38 39 40 long lat 1000 1200 1400 1600 1800 Precip(mm/year)a b Figure 7.4: The spatial distribution of daily mean (a) and annual mean rainfall estimated from long term data (1994-2009). Final rainfall estimates Abera et al. but can be corrected Aberaetal,2016a
  53. 53. !53 We divide the UBN basin into 402 subbasins and channel links as shown in figure 7.2. This spatial partitioning may not be the finest scale possible, however, considering the size of the basin, it can be considered an acceptable compromise to capture the water budget spatial variability. ADIGE: Rainfall-runoff Figure 7.3: Workflow with a list of NewAge components (in white), and remote sensing data processing parts (gray shaded, not yet included in JGrass-NewAGE but performed with R tools) used to derive the water budget of UBN. It does not include the components used for the validation and verification processes. The Modelling Solution calibration phase Abera et al. Schemes of work Aberaetal,inpreparation,2016c
  54. 54. !54 Discharges Abera et al. At daily time scale Aberaetal,inpreparation,2016c
  55. 55. !55 Abera et al. ET (spatial) Aberaetal,inpreparation,2016c
  56. 56. !56 Abera et al. The water budget (spatial) Aberaetal,inpreparation,2016c
  57. 57. !57 JGRASS-NEWAGE MODEL SYSTEM AND SATELLITE DATA 0 100 200 Precip[mm/month] −100 0 100 01 02 03 04 05 06 07 08 09 10 11 12 Months Fluxes(Q,ET,S)[mm/month] ET Q S Figure 7.16: Basin scale long term monthly mean Water budget components based on estimates from 1994 to 2009. It shows the relative share of the three components (Q, ET and S) of the total available water J. 160 Abera et al. The water budget (temporal) Aberaetal,inpreparation,2016c
  58. 58. !58 based on the NewAge modelling at subbasin scale, and GRACE grid resolution of 10 . Due to the possible high leakage error introduced at high spatial resolution (Swenson and Wahr, 2006), statistical comparison at subbasin level is not performed. However, focusing on maps of the sample months, some level of similar spatial and temporal pattern is revealed (figure 7.12). −100 0 100 200 2004 2005 2006 2007 2008 2009 2010 Date TWSC(mm/month) NewAge GRACE Correlation = 0.84 Figure 7.11: Comparison between basin scale NewAge ds/dt and GRACE TWSC from 2004-2009 at monthly time step. 7.5.2 Water budget closure The water budget components (J, ET, Q, ds/dt) of 402 subbasin of UBN is simulated for duration of 1994-2009 at daily time series. Figure 7.13 is long term monthly mean water JGrassNewAGE—GRACE comparison Abera et al. Storage variations Aberaetal,inpreparation,2016c
  59. 59. !59 Adige (12000 Km2) This is a work in progress Abera et al. Ongoing
  60. 60. !60 Ongoing Forecasting positions arm courtesy of Stefano Tasin Abera et al.
  61. 61. !61 Abera et al. Infos Introduction to JGrass-NewAGE http://abouthydrology.blogspot.it/2015/03/jgrass-newage-essentials.html Documentation http://geoframe.blogspot.it/
  62. 62. !62 Find this presentation at http://abouthydrology.blogspot.com Ulrici,2000? Other material at Questions ? R. Rigon http://www.slideshare.net/GEOFRAMEcafe/parma-20160517-jgrassnewage-some- about-the-state-of-art

×