Meteo I/O Introduction

  • 761 views
Uploaded on

Explains the features of the Meteo I/O library dedicated to treat meteo data

Explains the features of the Meteo I/O library dedicated to treat meteo data

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
761
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. MeteoIO A meteo data I/O library
  • 2. Before MeteoIO... • Each application doing its IO – Insufficient robustness – IO functions spread out all over the code – Input data very strictly formatted, several steps in preparing the data – Hard to adapt to new protocols/formats →IO a source of weakness... 2
  • 3. MeteoIO: goals • Remove all IO functions out of the physics engine • Make IO easy to use • Make IO robust • Make IO flexible • Greatly simplify data preparation • Allow unmonitored, automatic operation • Allow end user to fine tune his IO... 3
  • 4. MeteoIO: ini files [Parameters] DEMSRC = ARC DEMFILE = ../input/surface-grids/zwischberg.dem LANDUSESRC = GRASS LANDUSEFILE = ../input/surface-grids/zwischberg.lus plugins METEOSRC = A3D METEOPATH = ../input/meteo SPECIALPTSSRC = A3D SPECIALPTSFILE = ../input/surface-grids/zwischberg.pts OUTPUT = ARC OUTPATH = ../output/ 4
  • 5. MeteoIO: primary data structures • MeteoData: • StationData: – Date object – Longitude – Ta – Latitude – Iswr – Altitude – Vw – Northing (metric grid) – Rh – Easting (metric grid) – Lwr – Name – Nswc – Supports equality comparison – Ts0 – P – ... – Supports equality comparison 5
  • 6. MeteoIO: example of use 6
  • 7. What is it doing? • Parsing io.ini • Loading plugins supporting requested data sources • Reading data (across files, network, protocols and supporting comments, different EOL, etc) • Converts stations' coordinates if necessary • Filtering data, marking invalid data as “no data” • Conversion of units to SI 7
  • 8. Example: output [i] meteoio/A3DIO.cc:86: Loading dynamic plugins: [i] Number of 2D meteo stations: 15 ---------- Station: 1 / 16 Name: Air Temperature: 266.25 This station has no name ---------- Station: 2 / 16 Name: ANET-ZER Air Temperature: 266.45 ---------- Station: 3 / 16 Name: ANET-VIS Air Temperature: 268.75 (etc) 8
  • 9. Data sources • Currently supported: – Meteo data: Alpine3D's Ascii, Boschung's XML, Gsn, Imis Oracle database, GeoTop – Grids: ARC, GRASS 9
  • 10. Writing a new IO plugin • Provide implementation of some or all of: ReadMeteoData (const Date_IO& dateStart, const Date_IO& dateEnd, std::vector< std::vector<MeteoData> >& vecMeteo, std::vector< std::vector<StationData> >& vecStation, const unsigned int& stationindex=IOUtils::npos) Read2DGrid (Grid2DObject& grid_out, const string& parameter="") ReadDEM (DEMObject& dem_out) ReadLanduse (Grid2DObject& landuse_out) ReadAssimilationData (const Date_IO& date_in, Grid2DObject& da_out) ReadSpecialPoints (CSpecialPTSArray& pts) Write2DGrid (const Grid2DObject& grid_in, const string& name="") 10
  • 11. Filters • Available: – Min, max – Max observed changed – No observed change • To be added: – Median Average Deviation • Special processing – Resampling – accumulation • Easy to add new filters 11
  • 12. MeteoIO: example of use II 12
  • 13. What does it change? • IO are now buffered -> efficient on slow access • Automatic resampling for missing data – Using linear interpolation between two neighbors – To be added: cyclic interpolation, etc 13
  • 14. Userland addition In the user's code: add a function checking that the input requirements are met (ie: nb of lwr, nb of ta, etc) 14
  • 15. Data structures/methods • Array, Array2D, Array3D – Using vectors for a safe and efficient implementation • Grid2DObject, Grid3DObject – Geolocalized array • DEMObject – Computes slope, aspect, normal's components (Corripio, Hicks, etc) 15
  • 16. MeteoIO: example of use III 16
  • 17. MeteoIO: example of use IV 17
  • 18. TA, 0 to 16° RH, 62% to 90% Gotthard, 2008­10­01 12:00, 19 stations 18
  • 19. Spatial interpolations • Available methods: – Std pressure – Constant value – Constant with lapse rate – IDW – IDW with lapse rate – For Rh, Td IDWK with lapse rate and back to Rh – Terrain influenced VW and DW • One method defined for only 1 data input, one for multiple data input (fallback) • Regression: – Linear regression for the moment, removes the worst data point if the regression is too poor 19
  • 20. Spatial interpolations 2 Planned additions: – Physical wind interpolation (quick Bernoulli resolution?) – More regression models (dynamically adjusted to the data) – Smarter choice of which method to use (dynamic, including data quality) 20
  • 21. Geographic projections • Conversion to/from WGS84 from any coordinate system – Through own implementation (CH1903, ...) – Through proj4 library • Conversion to/from local grid MapProj local_proj(lat_ref, long_ref); local_proj.convert_from_WGS84(latitude, longitude, easting, northing); 21
  • 22. MeteoIO: on the side • Date handling (Date_IO class) • Config files handling (ConfigReader class) • Data storage (Arrays & grids classes, sparse matrix (coming)) • Ported to Pop-C++ for parallel execution • Application specific IOs classes 22
  • 23. Thank you! Mathias Bavay (SLF) Thomas Egger (Egger Consulting) Laurent Winkler (HES-SO) Florian Hof (SLF) Moustapha Mbengue (SLF) Gael Rosset (HES-SO) Julien Jeanneret (HES-SO) Nora Helbig (SLF) 23