Shows the new integrated toolchain: GSN-Meteo I/0-GEOtop for real-time simulations

1. Real-Time Toolchain
GSN - MeteoIO - GEOtop
Thomas Egger
Lausanne, 10.12.2009

2. GOAL
● Collect from Sensorscope stations
● Channel data through GSN
● Feed GEOtop with real-time data from GSN
● Publish real-time simulation results

3. Step-by-Step
● Making GEOtop a real-time model
● GEOtop & MeteoIO interaction
● Toolchain (data acquisition &
simulation)
● Using GSN Webservice (SOAP/WSDL)
● Demonstration
● Outlook

4. GEOtop Initial Condition
● Strictly linear processing
● No usable start, pause and resume
mechanisms
● Difficult configuration of parameters
● Meticulous, time-consuming preparation
of data necessary, no safeguards
(segfaults)
● How to make a model that can be used
reliably with real-time data?

5. GEOtop's Dark Side
● Time required for an expert to run a reference
simulation: 4 days
● Extremely complicated internal data structures:
tricky even for senior developers
● No checks on parameters files: when format changes
-> crashes without any hint of what is wrong
● Moving target: heavy development occurring on
GEOtop
➢ requires an expert to run, no automatic,
unattended execution possible

6. GEOtop Goals
● Start, pause and resume mechanism
● Use different data sources, without manual
preparation (robustness, automatic filtering)
● Easier configuration of simulation
● Increased stability (unattended run)

7. Real-Time GEOtop
● Start, pause and resume through recovery
mechanism
● Access to different data sources, data cleaning &
filtering through MeteoIO
● Command-Line-Interface for configuration of
simulation
● Making code more robust through debugging
(segmentation faults, leaking, uninitialized
variables)

8. GEOtop Recovery Mechanism
● Save status of simulation after each timestep
● Recovery files in human readable form
● Restore status for a given timestep
● Pitfalls
– Global variables (in C: extern AND static vars)
– Precision of output
fprintf(f, “%.*f”, (int)LDBL_DIG, doublevalue);

9. GEOtop & MeteoIO (1)
● MeteoIO is a library that provides
– access to different meteorological data sources
through different plugins (meteo data, 2D/3D grids)
– advanced methods for data cleaning & filtering as
well as spatial interpolation algorithms
● MeteoIO is written in C++, thus wrappers were
necessary to “translate” data
● Two MeteoIO plugins developed
– GSNIO (accessing GSN data through web service)
– GeotopIO (parsing CSV files)

10. GEOtop & MeteoIO (2)

11. Mixing C/C++
● Using compatible compilers (e.g. gcc)
● Declare linkage for functions (extern keyword)
#include <iostream>
extern "C" int print(int i, double d)
{
std::cout << "i = " << i << ", d = " << d;
}
● Concept of classes compatible with structs
● Deal with C++ exceptions

12. Toolchain (1)
1.Meteo stations transmit data to
Sensorscope servers
2.GSN fetches data by listening to serial
data stream, using web service or CSV
wrapper
3.GSN triggers a simulation
(GEOtop/A3D) if enough data present
for next timestep
4.GEOtop/A3D access the data from GSN
through the MeteoIO library

13. Toolchain (2)
Sensorscope weather
stations La Fouly
Sensorscope database GSN Network Simulation server
Lausanne Lausanne running GEOtop/A3D
Lausanne

14. Toolchain (3)
Simulation server
running GEOtop/A3D
Lausanne
[3] RSH
Sensorscope weather
stations La Fouly
[4] SOAP
[1] GPRS
File IO
[2] CSV, [5] SFTP
Serial, WS Post-processing
Sensorscope database GSN Network
Lausanne Lausanne

15. What is GSN?
● GSN is a data acquisition network (JAVA)
● It is build around the concept of “sensors”
– Real sensors (e.g. thermometer)
– Virtual sensors (e.g. averaging filter, GEOtop)
● Many wrappers for different sensors exist

16. Sensorscope GSN Interaction
● For GSN Sensorscope stations are sensors
● CSV Wrapper
Importing data by parsing CSV files
● Serial Listener
Listening to data packages as they come on a a
TCP port (event based messaging)
● Future: Web service or database access

17. Web Services (1)
● By using web services an application can publish
its functions on the internet
● Definition of what functions are available in web
service description language (WSDL) in XML
● XML used to code and decode data and SOAP to
transport it (over HTTP)
● Heterogeneous environments can easily be
connected

18. Web Services (2)
● Machine-to-Machine interface (automation)
● 2 ways of development:
– bottom-up (start with implementation, generate
WSDL)
– top-down (start with design, generate skeleton)
● Similar technologies: RMI, CORBA, XML-RPC

19. GSN Web Service
● MeteoIO plugin GSNIO accesses data on GSN
servers by means of a web service
● Simple access functions:
– getSensors()
– getSensorInfo(stationname)
– getSensorLocation(stationname)
– getMeteoData(station, startDate, endDate)
● GSNWebService.wsdl

20. MeteoIO & GSN Webservice
● wsdl2h generates C++ header file
● soapcpp2 generates C++ stubs
● Incorporated into plugin GSNIO
● Example:
Accessing GSN webservice through MeteoIO

21. Demonstration
● Retrieving data through MeteoIO
– GEOtopIO plugin
– GSNIO plugin
● Script run.pl
(Invokes A3D/GEOtop and post-processing)
● GSN webpage displaying results of sensors

22. Outlook
● GSN web interface for starting and running
models
● GSN support for reading 2D Grids (DEM)
● Session support (so different users can execute
simulation at same time) in both GEOtop and
A3D