MeteoIO


A meteo data I/O library
Before MeteoIO...
• Each application doing its IO
  – Insufficient robustness
  – IO functions spread out all over the cod...
MeteoIO: goals
• Remove all IO functions out of the
  physics engine
• Make IO easy to use
• Make IO robust
• Make IO flex...
MeteoIO: ini files
[Parameters]
DEMSRC            = ARC
DEMFILE           = ../input/surface-grids/zwischberg.dem


LANDUS...
MeteoIO: primary data structures
• MeteoData:                       • StationData:
  – Date object                      – ...
MeteoIO: example of use




                          6
What is it doing?
• Parsing io.ini
• Loading plugins supporting requested data sources
• Reading data (across files, netwo...
Example: output
[i] meteoio/A3DIO.cc:86: Loading dynamic plugins:
[i] Number of 2D meteo stations: 15
---------- Station: ...
Data sources
• Currently supported:
  – Meteo data: Alpine3D's Ascii, Boschung's
    XML, Gsn, Imis Oracle database, GeoTo...
Writing a new IO plugin
• Provide implementation of some or all of:
ReadMeteoData (const Date_IO& dateStart, const Date_IO...
Filters
• Available:
  – Min, max
  – Max observed changed
  – No observed change
• To be added:
  – Median Average Deviat...
MeteoIO: example of use II




                             12
What does it change?
• IO are now buffered -> efficient on slow
  access
• Automatic resampling for missing data
  – Using...
Userland addition
In the user's code: add a function checking
     that the input requirements are met
         (ie: nb of...
Data structures/methods
• Array, Array2D, Array3D
  – Using vectors for a safe and efficient
    implementation
• Grid2DOb...
MeteoIO: example of use III




                              16
MeteoIO: example of use IV




                             17
TA, 0 to 16°                    RH, 62% to 90%




  Gotthard, 2008­10­01 12:00, 19 stations        18
Spatial interpolations
• Available methods:
   – Std pressure
   – Constant value
   – Constant with lapse rate
   – IDW
 ...
Spatial interpolations 2
Planned additions:
  – Physical wind interpolation (quick Bernoulli
    resolution?)
  – More reg...
Geographic projections
• Conversion to/from WGS84 from any
  coordinate system
  – Through own implementation (CH1903, ......
MeteoIO: on the side
• Date handling (Date_IO class)
• Config files handling (ConfigReader
  class)
• Data storage (Arrays...
Thank you!
       Mathias Bavay (SLF)
Thomas Egger (Egger Consulting)
   Laurent Winkler (HES-SO)
        Florian Hof (SLF...
Upcoming SlideShare
Loading in …5
×

Meteo I/O Introduction

1,108 views

Published on

Explains the features of the Meteo I/O library dedicated to treat meteo data

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
1,108
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
11
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Meteo I/O Introduction

  1. 1. MeteoIO A meteo data I/O library
  2. 2. Before MeteoIO... • Each application doing its IO – Insufficient robustness – IO functions spread out all over the code – Input data very strictly formatted, several steps in preparing the data – Hard to adapt to new protocols/formats →IO a source of weakness... 2
  3. 3. MeteoIO: goals • Remove all IO functions out of the physics engine • Make IO easy to use • Make IO robust • Make IO flexible • Greatly simplify data preparation • Allow unmonitored, automatic operation • Allow end user to fine tune his IO... 3
  4. 4. MeteoIO: ini files [Parameters] DEMSRC = ARC DEMFILE = ../input/surface-grids/zwischberg.dem LANDUSESRC = GRASS LANDUSEFILE = ../input/surface-grids/zwischberg.lus plugins METEOSRC = A3D METEOPATH = ../input/meteo SPECIALPTSSRC = A3D SPECIALPTSFILE = ../input/surface-grids/zwischberg.pts OUTPUT = ARC OUTPATH = ../output/ 4
  5. 5. MeteoIO: primary data structures • MeteoData: • StationData: – Date object – Longitude – Ta – Latitude – Iswr – Altitude – Vw – Northing (metric grid) – Rh – Easting (metric grid) – Lwr – Name – Nswc – Supports equality comparison – Ts0 – P – ... – Supports equality comparison 5
  6. 6. MeteoIO: example of use 6
  7. 7. What is it doing? • Parsing io.ini • Loading plugins supporting requested data sources • Reading data (across files, network, protocols and supporting comments, different EOL, etc) • Converts stations' coordinates if necessary • Filtering data, marking invalid data as “no data” • Conversion of units to SI 7
  8. 8. Example: output [i] meteoio/A3DIO.cc:86: Loading dynamic plugins: [i] Number of 2D meteo stations: 15 ---------- Station: 1 / 16 Name: Air Temperature: 266.25 This station has no name ---------- Station: 2 / 16 Name: ANET-ZER Air Temperature: 266.45 ---------- Station: 3 / 16 Name: ANET-VIS Air Temperature: 268.75 (etc) 8
  9. 9. Data sources • Currently supported: – Meteo data: Alpine3D's Ascii, Boschung's XML, Gsn, Imis Oracle database, GeoTop – Grids: ARC, GRASS 9
  10. 10. Writing a new IO plugin • Provide implementation of some or all of: ReadMeteoData (const Date_IO& dateStart, const Date_IO& dateEnd, std::vector< std::vector<MeteoData> >& vecMeteo, std::vector< std::vector<StationData> >& vecStation, const unsigned int& stationindex=IOUtils::npos) Read2DGrid (Grid2DObject& grid_out, const string& parameter="") ReadDEM (DEMObject& dem_out) ReadLanduse (Grid2DObject& landuse_out) ReadAssimilationData (const Date_IO& date_in, Grid2DObject& da_out) ReadSpecialPoints (CSpecialPTSArray& pts) Write2DGrid (const Grid2DObject& grid_in, const string& name="") 10
  11. 11. Filters • Available: – Min, max – Max observed changed – No observed change • To be added: – Median Average Deviation • Special processing – Resampling – accumulation • Easy to add new filters 11
  12. 12. MeteoIO: example of use II 12
  13. 13. What does it change? • IO are now buffered -> efficient on slow access • Automatic resampling for missing data – Using linear interpolation between two neighbors – To be added: cyclic interpolation, etc 13
  14. 14. Userland addition In the user's code: add a function checking that the input requirements are met (ie: nb of lwr, nb of ta, etc) 14
  15. 15. Data structures/methods • Array, Array2D, Array3D – Using vectors for a safe and efficient implementation • Grid2DObject, Grid3DObject – Geolocalized array • DEMObject – Computes slope, aspect, normal's components (Corripio, Hicks, etc) 15
  16. 16. MeteoIO: example of use III 16
  17. 17. MeteoIO: example of use IV 17
  18. 18. TA, 0 to 16° RH, 62% to 90% Gotthard, 2008­10­01 12:00, 19 stations 18
  19. 19. Spatial interpolations • Available methods: – Std pressure – Constant value – Constant with lapse rate – IDW – IDW with lapse rate – For Rh, Td IDWK with lapse rate and back to Rh – Terrain influenced VW and DW • One method defined for only 1 data input, one for multiple data input (fallback) • Regression: – Linear regression for the moment, removes the worst data point if the regression is too poor 19
  20. 20. Spatial interpolations 2 Planned additions: – Physical wind interpolation (quick Bernoulli resolution?) – More regression models (dynamically adjusted to the data) – Smarter choice of which method to use (dynamic, including data quality) 20
  21. 21. Geographic projections • Conversion to/from WGS84 from any coordinate system – Through own implementation (CH1903, ...) – Through proj4 library • Conversion to/from local grid MapProj local_proj(lat_ref, long_ref); local_proj.convert_from_WGS84(latitude, longitude, easting, northing); 21
  22. 22. MeteoIO: on the side • Date handling (Date_IO class) • Config files handling (ConfigReader class) • Data storage (Arrays & grids classes, sparse matrix (coming)) • Ported to Pop-C++ for parallel execution • Application specific IOs classes 22
  23. 23. Thank you! Mathias Bavay (SLF) Thomas Egger (Egger Consulting) Laurent Winkler (HES-SO) Florian Hof (SLF) Moustapha Mbengue (SLF) Gael Rosset (HES-SO) Julien Jeanneret (HES-SO) Nora Helbig (SLF) 23

×