Toolchain for real-time simulations: GSN-MeteoIO-GEOtop


Published on

Shows the new integrated toolchain: GSN-Meteo I/0-GEOtop for real-time simulations

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Toolchain for real-time simulations: GSN-MeteoIO-GEOtop

  1. 1. Real-Time Toolchain GSN - MeteoIO - GEOtop Thomas Egger Lausanne, 10.12.2009
  2. 2. GOAL ● Collect from Sensorscope stations ● Channel data through GSN ● Feed GEOtop with real-time data from GSN ● Publish real-time simulation results
  3. 3. Step-by-Step ● Making GEOtop a real-time model ● GEOtop & MeteoIO interaction ● Toolchain (data acquisition & simulation) ● Using GSN Webservice (SOAP/WSDL) ● Demonstration ● Outlook
  4. 4. GEOtop Initial Condition ● Strictly linear processing ● No usable start, pause and resume mechanisms ● Difficult configuration of parameters ● Meticulous, time-consuming preparation of data necessary, no safeguards (segfaults) ● How to make a model that can be used reliably with real-time data?
  5. 5. GEOtop's Dark Side ● Time required for an expert to run a reference simulation: 4 days ● Extremely complicated internal data structures: tricky even for senior developers ● No checks on parameters files: when format changes -> crashes without any hint of what is wrong ● Moving target: heavy development occurring on GEOtop ➢ requires an expert to run, no automatic, unattended execution possible
  6. 6. GEOtop Goals ● Start, pause and resume mechanism ● Use different data sources, without manual preparation (robustness, automatic filtering) ● Easier configuration of simulation ● Increased stability (unattended run)
  7. 7. Real-Time GEOtop ● Start, pause and resume through recovery mechanism ● Access to different data sources, data cleaning & filtering through MeteoIO ● Command-Line-Interface for configuration of simulation ● Making code more robust through debugging (segmentation faults, leaking, uninitialized variables)
  8. 8. GEOtop Recovery Mechanism ● Save status of simulation after each timestep ● Recovery files in human readable form ● Restore status for a given timestep ● Pitfalls – Global variables (in C: extern AND static vars) – Precision of output fprintf(f, “%.*f”, (int)LDBL_DIG, doublevalue);
  9. 9. GEOtop & MeteoIO (1) ● MeteoIO is a library that provides – access to different meteorological data sources through different plugins (meteo data, 2D/3D grids) – advanced methods for data cleaning & filtering as well as spatial interpolation algorithms ● MeteoIO is written in C++, thus wrappers were necessary to “translate” data ● Two MeteoIO plugins developed – GSNIO (accessing GSN data through web service) – GeotopIO (parsing CSV files)
  10. 10. GEOtop & MeteoIO (2)
  11. 11. Mixing C/C++ ● Using compatible compilers (e.g. gcc) ● Declare linkage for functions (extern keyword) #include <iostream> extern "C" int print(int i, double d) { std::cout << "i = " << i << ", d = " << d; } ● Concept of classes compatible with structs ● Deal with C++ exceptions
  12. 12. Toolchain (1) 1.Meteo stations transmit data to Sensorscope servers 2.GSN fetches data by listening to serial data stream, using web service or CSV wrapper 3.GSN triggers a simulation (GEOtop/A3D) if enough data present for next timestep 4.GEOtop/A3D access the data from GSN through the MeteoIO library
  13. 13. Toolchain (2) Sensorscope weather stations La Fouly Sensorscope database GSN Network Simulation server Lausanne Lausanne running GEOtop/A3D Lausanne
  14. 14. Toolchain (3) Simulation server running GEOtop/A3D Lausanne [3] RSH Sensorscope weather stations La Fouly [4] SOAP [1] GPRS File IO [2] CSV, [5] SFTP Serial, WS Post-processing Sensorscope database GSN Network Lausanne Lausanne
  15. 15. What is GSN? ● GSN is a data acquisition network (JAVA) ● It is build around the concept of “sensors” – Real sensors (e.g. thermometer) – Virtual sensors (e.g. averaging filter, GEOtop) ● Many wrappers for different sensors exist
  16. 16. Sensorscope GSN Interaction ● For GSN Sensorscope stations are sensors ● CSV Wrapper Importing data by parsing CSV files ● Serial Listener Listening to data packages as they come on a a TCP port (event based messaging) ● Future: Web service or database access
  17. 17. Web Services (1) ● By using web services an application can publish its functions on the internet ● Definition of what functions are available in web service description language (WSDL) in XML ● XML used to code and decode data and SOAP to transport it (over HTTP) ● Heterogeneous environments can easily be connected
  18. 18. Web Services (2) ● Machine-to-Machine interface (automation) ● 2 ways of development: – bottom-up (start with implementation, generate WSDL) – top-down (start with design, generate skeleton) ● Similar technologies: RMI, CORBA, XML-RPC
  19. 19. GSN Web Service ● MeteoIO plugin GSNIO accesses data on GSN servers by means of a web service ● Simple access functions: – getSensors() – getSensorInfo(stationname) – getSensorLocation(stationname) – getMeteoData(station, startDate, endDate) ● GSNWebService.wsdl
  20. 20. MeteoIO & GSN Webservice ● wsdl2h generates C++ header file ● soapcpp2 generates C++ stubs ● Incorporated into plugin GSNIO ● Example: Accessing GSN webservice through MeteoIO
  21. 21. Demonstration ● Retrieving data through MeteoIO – GEOtopIO plugin – GSNIO plugin ● Script (Invokes A3D/GEOtop and post-processing) ● GSN webpage displaying results of sensors
  22. 22. Outlook ● GSN web interface for starting and running models ● GSN support for reading 2D Grids (DEM) ● Session support (so different users can execute simulation at same time) in both GEOtop and A3D