SlideShare a Scribd company logo
Real-Time Toolchain
GSN - MeteoIO - GEOtop

       Thomas Egger
    Lausanne, 10.12.2009
GOAL
●   Collect from Sensorscope stations
●   Channel data through GSN
●   Feed GEOtop with real-time data from GSN
●   Publish real-time simulation results
Step-by-Step
●   Making GEOtop a real-time model
●   GEOtop & MeteoIO interaction
●   Toolchain (data acquisition &
    simulation)
●   Using GSN Webservice (SOAP/WSDL)
●   Demonstration
●   Outlook
GEOtop Initial Condition
●   Strictly linear processing
●   No usable start, pause and resume
    mechanisms
●   Difficult configuration of parameters
●   Meticulous, time-consuming preparation
    of data necessary, no safeguards
    (segfaults)
●   How to make a model that can be used
    reliably with real-time data?
GEOtop's Dark Side
●   Time required for an expert to run a reference
    simulation: 4 days
●   Extremely complicated internal data structures:
    tricky even for senior developers
●   No checks on parameters files: when format changes
    -> crashes without any hint of what is wrong
●   Moving target: heavy development occurring on
    GEOtop
➢   requires an expert to run, no automatic,
    unattended execution possible
GEOtop Goals
●   Start, pause and resume mechanism
●   Use different data sources, without manual
    preparation (robustness, automatic filtering)
●   Easier configuration of simulation
●   Increased stability (unattended run)
Real-Time GEOtop
●   Start, pause and resume through recovery
    mechanism
●   Access to different data sources, data cleaning &
    filtering through MeteoIO
●   Command-Line-Interface for configuration of
    simulation
●   Making code more robust through debugging
    (segmentation faults, leaking, uninitialized
    variables)
GEOtop Recovery Mechanism
●   Save status of simulation after each timestep
●   Recovery files in human readable form
●   Restore status for a given timestep
●   Pitfalls
     –   Global variables (in C: extern AND static vars)
     –   Precision of output
         fprintf(f, “%.*f”, (int)LDBL_DIG, doublevalue);
GEOtop & MeteoIO (1)
●   MeteoIO is a library that provides
    –   access to different meteorological data sources
        through different plugins (meteo data, 2D/3D grids)
    –   advanced methods for data cleaning & filtering as
        well as spatial interpolation algorithms
●   MeteoIO is written in C++, thus wrappers were
    necessary to “translate” data
●   Two MeteoIO plugins developed
    –   GSNIO (accessing GSN data through web service)
    –   GeotopIO (parsing CSV files)
GEOtop & MeteoIO (2)
Mixing C/C++
●   Using compatible compilers (e.g. gcc)
●   Declare linkage for functions (extern keyword)
    #include <iostream>
    extern "C" int print(int i, double d)
    {
        std::cout << "i = " << i << ", d = " << d;
    }


●   Concept of classes compatible with structs
●   Deal with C++ exceptions
Toolchain (1)
1.Meteo stations transmit data to
 Sensorscope servers
2.GSN fetches data by listening to serial
 data stream, using web service or CSV
 wrapper
3.GSN triggers a simulation
 (GEOtop/A3D) if enough data present
 for next timestep
4.GEOtop/A3D access the data from GSN
  through the MeteoIO library
Toolchain (2)


Sensorscope weather
  stations La Fouly




Sensorscope database   GSN Network     Simulation server
      Lausanne          Lausanne     running GEOtop/A3D
                                           Lausanne
Toolchain (3)
                                                    Simulation server
                                                  running GEOtop/A3D
                                                        Lausanne
                                    [3] RSH

  Sensorscope weather
    stations La Fouly
                                       [4] SOAP
[1] GPRS
                                                              File IO



                [2] CSV,               [5] SFTP
                Serial, WS                          Post-processing
 Sensorscope database        GSN Network
       Lausanne               Lausanne
What is GSN?
●   GSN is a data acquisition network (JAVA)
●   It is build around the concept of “sensors”
    –   Real sensors (e.g. thermometer)
    –   Virtual sensors (e.g. averaging filter, GEOtop)
●   Many wrappers for different sensors exist
Sensorscope GSN Interaction
●   For GSN Sensorscope stations are sensors
●   CSV Wrapper
    Importing data by parsing CSV files
●   Serial Listener
    Listening to data packages as they come on a a
    TCP port (event based messaging)
●   Future: Web service or database access
Web Services (1)
●   By using web services an application can publish
    its functions on the internet
●   Definition of what functions are available in web
    service description language (WSDL) in XML
●   XML used to code and decode data and SOAP to
    transport it (over HTTP)
●   Heterogeneous environments can easily be
    connected
Web Services (2)
●   Machine-to-Machine interface (automation)
●   2 ways of development:
    –   bottom-up (start with implementation, generate
        WSDL)
    –   top-down (start with design, generate skeleton)


●   Similar technologies: RMI, CORBA, XML-RPC
GSN Web Service
●   MeteoIO plugin GSNIO accesses data on GSN
    servers by means of a web service
●   Simple access functions:
    –   getSensors()
    –   getSensorInfo(stationname)
    –   getSensorLocation(stationname)
    –   getMeteoData(station, startDate, endDate)

●   GSNWebService.wsdl
MeteoIO & GSN Webservice
●   wsdl2h generates C++ header file
●   soapcpp2 generates C++ stubs
●   Incorporated into plugin GSNIO

●   Example:
    Accessing GSN webservice through MeteoIO
Demonstration
●   Retrieving data through MeteoIO
    –   GEOtopIO plugin
    –   GSNIO plugin
●   Script run.pl
    (Invokes A3D/GEOtop and post-processing)
●   GSN webpage displaying results of sensors
Outlook
●   GSN web interface for starting and running
    models
●   GSN support for reading 2D Grids (DEM)
●   Session support (so different users can execute
    simulation at same time) in both GEOtop and
    A3D

More Related Content

What's hot

Quantum Computing Fundamentals via OO
Quantum Computing Fundamentals via OOQuantum Computing Fundamentals via OO
Quantum Computing Fundamentals via OO
Carl Belle
 
Learning Erlang (from a Prolog dropout's perspective)
Learning Erlang (from a Prolog dropout's perspective)Learning Erlang (from a Prolog dropout's perspective)
Learning Erlang (from a Prolog dropout's perspective)
elliando dias
 
Quantum Computing and Qiskit
Quantum Computing and QiskitQuantum Computing and Qiskit
Quantum Computing and Qiskit
Pooja Mistry
 
Advanced Tracing features using GDB and LTTng
Advanced Tracing features using GDB and LTTngAdvanced Tracing features using GDB and LTTng
Advanced Tracing features using GDB and LTTng
marckhouzam
 
Introsort or introspective sort
Introsort or introspective sortIntrosort or introspective sort
Introsort or introspective sort
Eftykhar Mahmud
 
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
OpenNebula Project
 
Chainer Update v1.8.0 -> v1.10.0+
Chainer Update v1.8.0 -> v1.10.0+Chainer Update v1.8.0 -> v1.10.0+
Chainer Update v1.8.0 -> v1.10.0+
Seiya Tokui
 
Running Multiple XORP Instances In One Box
Running Multiple XORP Instances In One BoxRunning Multiple XORP Instances In One Box
Running Multiple XORP Instances In One Box
Jiaqing Du
 
Slide smallfiles
Slide smallfilesSlide smallfiles
Slide smallfiles
rledisez
 
cloud_futures_2.0_Papazachos
cloud_futures_2.0_Papazachoscloud_futures_2.0_Papazachos
cloud_futures_2.0_Papazachos
George Spyrou
 
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
Storti Mario
 
Real-time Debugging using GDB Tracepoints and other Eclipse features
Real-time Debugging using GDB Tracepoints and other Eclipse features Real-time Debugging using GDB Tracepoints and other Eclipse features
Real-time Debugging using GDB Tracepoints and other Eclipse features
marckhouzam
 
Peer Sim (P2P network)
Peer Sim (P2P network)Peer Sim (P2P network)
Peer Sim (P2P network)
Sijo Emmanuel
 
Working with OpenStreetMap using Apache Spark and Geotrellis
Working with OpenStreetMap using Apache Spark and GeotrellisWorking with OpenStreetMap using Apache Spark and Geotrellis
Working with OpenStreetMap using Apache Spark and Geotrellis
Rob Emanuele
 
Peer sim (p2p network)
Peer sim (p2p network)Peer sim (p2p network)
Peer sim (p2p network)
Hein Min Htike
 
Real time debugging: using non-intrusive tracepoints to debug live systems
Real time debugging: using non-intrusive tracepoints to debug live systemsReal time debugging: using non-intrusive tracepoints to debug live systems
Real time debugging: using non-intrusive tracepoints to debug live systems
marckhouzam
 
Introduction to Garbage Collection
Introduction to Garbage CollectionIntroduction to Garbage Collection
Introduction to Garbage Collection
Artur Mkrtchyan
 
QUANTUM COMP 22
QUANTUM COMP 22QUANTUM COMP 22
QUANTUM COMP 22
Tejasri Jampani
 
Intro to cassandra
Intro to cassandraIntro to cassandra
Intro to cassandra
JWORKS powered by Ordina
 
Optimizing High Performance Big Data Cancer Workflows
Optimizing High Performance Big Data Cancer WorkflowsOptimizing High Performance Big Data Cancer Workflows
Optimizing High Performance Big Data Cancer Workflows
Ivan Jimenez
 

What's hot (20)

Quantum Computing Fundamentals via OO
Quantum Computing Fundamentals via OOQuantum Computing Fundamentals via OO
Quantum Computing Fundamentals via OO
 
Learning Erlang (from a Prolog dropout's perspective)
Learning Erlang (from a Prolog dropout's perspective)Learning Erlang (from a Prolog dropout's perspective)
Learning Erlang (from a Prolog dropout's perspective)
 
Quantum Computing and Qiskit
Quantum Computing and QiskitQuantum Computing and Qiskit
Quantum Computing and Qiskit
 
Advanced Tracing features using GDB and LTTng
Advanced Tracing features using GDB and LTTngAdvanced Tracing features using GDB and LTTng
Advanced Tracing features using GDB and LTTng
 
Introsort or introspective sort
Introsort or introspective sortIntrosort or introspective sort
Introsort or introspective sort
 
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
OpenNebulaConf2018 - OpenNebula and LXD Containers - Rubén S. Montero - OpenN...
 
Chainer Update v1.8.0 -> v1.10.0+
Chainer Update v1.8.0 -> v1.10.0+Chainer Update v1.8.0 -> v1.10.0+
Chainer Update v1.8.0 -> v1.10.0+
 
Running Multiple XORP Instances In One Box
Running Multiple XORP Instances In One BoxRunning Multiple XORP Instances In One Box
Running Multiple XORP Instances In One Box
 
Slide smallfiles
Slide smallfilesSlide smallfiles
Slide smallfiles
 
cloud_futures_2.0_Papazachos
cloud_futures_2.0_Papazachoscloud_futures_2.0_Papazachos
cloud_futures_2.0_Papazachos
 
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
Advances in the Solution of NS Eqs. in GPGPU Hardware. Second order scheme an...
 
Real-time Debugging using GDB Tracepoints and other Eclipse features
Real-time Debugging using GDB Tracepoints and other Eclipse features Real-time Debugging using GDB Tracepoints and other Eclipse features
Real-time Debugging using GDB Tracepoints and other Eclipse features
 
Peer Sim (P2P network)
Peer Sim (P2P network)Peer Sim (P2P network)
Peer Sim (P2P network)
 
Working with OpenStreetMap using Apache Spark and Geotrellis
Working with OpenStreetMap using Apache Spark and GeotrellisWorking with OpenStreetMap using Apache Spark and Geotrellis
Working with OpenStreetMap using Apache Spark and Geotrellis
 
Peer sim (p2p network)
Peer sim (p2p network)Peer sim (p2p network)
Peer sim (p2p network)
 
Real time debugging: using non-intrusive tracepoints to debug live systems
Real time debugging: using non-intrusive tracepoints to debug live systemsReal time debugging: using non-intrusive tracepoints to debug live systems
Real time debugging: using non-intrusive tracepoints to debug live systems
 
Introduction to Garbage Collection
Introduction to Garbage CollectionIntroduction to Garbage Collection
Introduction to Garbage Collection
 
QUANTUM COMP 22
QUANTUM COMP 22QUANTUM COMP 22
QUANTUM COMP 22
 
Intro to cassandra
Intro to cassandraIntro to cassandra
Intro to cassandra
 
Optimizing High Performance Big Data Cancer Workflows
Optimizing High Performance Big Data Cancer WorkflowsOptimizing High Performance Big Data Cancer Workflows
Optimizing High Performance Big Data Cancer Workflows
 

Similar to Toolchain for real-time simulations: GSN-MeteoIO-GEOtop

BWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 PresentationBWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 Presentation
lilyco
 
Monitoring federation open stack infrastructure
Monitoring federation open stack infrastructureMonitoring federation open stack infrastructure
Monitoring federation open stack infrastructure
Fernando Lopez Aguilar
 
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado BlascoDSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
Deltares
 
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
innov-acts-ltd
 
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
Guglielmo Iozzia
 
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaPrometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Sridhar Kumar N
 
Lecture OSSIM
Lecture OSSIM Lecture OSSIM
Lecture OSSIM
Margherita Di Leo
 
Meteo I/O Introduction
Meteo I/O IntroductionMeteo I/O Introduction
Meteo I/O Introduction
Riccardo Rigon
 
Sector Sphere 2009
Sector Sphere 2009Sector Sphere 2009
Sector Sphere 2009
lilyco
 
sector-sphere
sector-spheresector-sphere
sector-sphere
xlight
 
Model-driven Network Management
Model-driven Network ManagementModel-driven Network Management
Model-driven Network Management
Anees Shaikh
 
PIMRC-2012, Sydney, Australia, 28 July, 2012
PIMRC-2012, Sydney, Australia, 28 July, 2012PIMRC-2012, Sydney, Australia, 28 July, 2012
PIMRC-2012, Sydney, Australia, 28 July, 2012
Charith Perera
 
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellitesDesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
Brenden Hogan
 
Realini go gps_foss4g2011_small2
Realini go gps_foss4g2011_small2Realini go gps_foss4g2011_small2
Realini go gps_foss4g2011_small2
masarunarazaki
 
IoTivity for Automotive IoT Interoperability
IoTivity for Automotive IoT InteroperabilityIoTivity for Automotive IoT Interoperability
IoTivity for Automotive IoT Interoperability
Samsung Open Source Group
 
goGPS (November 2013)
goGPS (November 2013)goGPS (November 2013)
goGPS (November 2013)
Eugenio Realini
 
Zabbix Monitoring Platform
Zabbix Monitoring Platform Zabbix Monitoring Platform
Zabbix Monitoring Platform
Seyedmajid Etehadi
 
Opencensus with prometheus and kubernetes
Opencensus with prometheus and kubernetesOpencensus with prometheus and kubernetes
Opencensus with prometheus and kubernetes
Jinwoong Kim
 
Network time sync solutions for security
Network time sync solutions for securityNetwork time sync solutions for security
Network time sync solutions for security
Mohd Amir
 
LTTng-UST: Efficient System-Wide User-Space Tracing
LTTng-UST: Efficient System-Wide User-Space TracingLTTng-UST: Efficient System-Wide User-Space Tracing
LTTng-UST: Efficient System-Wide User-Space Tracing
Christian Babeux
 

Similar to Toolchain for real-time simulations: GSN-MeteoIO-GEOtop (20)

BWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 PresentationBWC Supercomputing 2008 Presentation
BWC Supercomputing 2008 Presentation
 
Monitoring federation open stack infrastructure
Monitoring federation open stack infrastructureMonitoring federation open stack infrastructure
Monitoring federation open stack infrastructure
 
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado BlascoDSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
DSD-INT 2015 - RSS Sentinel Toolbox - J. Manuel Delgado Blasco
 
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
H2020 finsec-ibm- aidan-shribman-finsec-skydive 260820
 
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
Building a data pipeline to ingest data into Hadoop in minutes using Streamse...
 
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,GrafanaPrometheus - Intro, CNCF, TSDB,PromQL,Grafana
Prometheus - Intro, CNCF, TSDB,PromQL,Grafana
 
Lecture OSSIM
Lecture OSSIM Lecture OSSIM
Lecture OSSIM
 
Meteo I/O Introduction
Meteo I/O IntroductionMeteo I/O Introduction
Meteo I/O Introduction
 
Sector Sphere 2009
Sector Sphere 2009Sector Sphere 2009
Sector Sphere 2009
 
sector-sphere
sector-spheresector-sphere
sector-sphere
 
Model-driven Network Management
Model-driven Network ManagementModel-driven Network Management
Model-driven Network Management
 
PIMRC-2012, Sydney, Australia, 28 July, 2012
PIMRC-2012, Sydney, Australia, 28 July, 2012PIMRC-2012, Sydney, Australia, 28 July, 2012
PIMRC-2012, Sydney, Australia, 28 July, 2012
 
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellitesDesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
DesignOfAnExtensibleTelemetry&CommandArcitectureForSmallSatellites
 
Realini go gps_foss4g2011_small2
Realini go gps_foss4g2011_small2Realini go gps_foss4g2011_small2
Realini go gps_foss4g2011_small2
 
IoTivity for Automotive IoT Interoperability
IoTivity for Automotive IoT InteroperabilityIoTivity for Automotive IoT Interoperability
IoTivity for Automotive IoT Interoperability
 
goGPS (November 2013)
goGPS (November 2013)goGPS (November 2013)
goGPS (November 2013)
 
Zabbix Monitoring Platform
Zabbix Monitoring Platform Zabbix Monitoring Platform
Zabbix Monitoring Platform
 
Opencensus with prometheus and kubernetes
Opencensus with prometheus and kubernetesOpencensus with prometheus and kubernetes
Opencensus with prometheus and kubernetes
 
Network time sync solutions for security
Network time sync solutions for securityNetwork time sync solutions for security
Network time sync solutions for security
 
LTTng-UST: Efficient System-Wide User-Space Tracing
LTTng-UST: Efficient System-Wide User-Space TracingLTTng-UST: Efficient System-Wide User-Space Tracing
LTTng-UST: Efficient System-Wide User-Space Tracing
 

More from Riccardo Rigon

Models for hazards mapping
Models for hazards mappingModels for hazards mapping
Models for hazards mapping
Riccardo Rigon
 
A short introduction to some hydrological extreme phenomena
A short introduction to some hydrological extreme phenomenaA short introduction to some hydrological extreme phenomena
A short introduction to some hydrological extreme phenomena
Riccardo Rigon
 
EvaporAzione
EvaporAzioneEvaporAzione
EvaporAzione
Riccardo Rigon
 
Francesco Serafin
Francesco Serafin Francesco Serafin
Francesco Serafin
Riccardo Rigon
 
Meledrio
MeledrioMeledrio
Meledrio
Riccardo Rigon
 
Lisbon talk for SteepStreams
Lisbon talk  for SteepStreamsLisbon talk  for SteepStreams
Lisbon talk for SteepStreams
Riccardo Rigon
 
Grids implementation
Grids implementationGrids implementation
Grids implementation
Riccardo Rigon
 
Grids
GridsGrids
Some photos from the field
Some photos from the fieldSome photos from the field
Some photos from the field
Riccardo Rigon
 
Virtual water fem 07032017
Virtual water fem 07032017Virtual water fem 07032017
Virtual water fem 07032017
Riccardo Rigon
 
Dalton Prize Lecture 2017 by Dani Or
Dalton Prize Lecture 2017 by Dani OrDalton Prize Lecture 2017 by Dani Or
Dalton Prize Lecture 2017 by Dani Or
Riccardo Rigon
 
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
Riccardo Rigon
 
The modern flood forecasting
The modern flood forecastingThe modern flood forecasting
The modern flood forecasting
Riccardo Rigon
 
La moderna previsione delle piene
La moderna previsione delle pieneLa moderna previsione delle piene
La moderna previsione delle piene
Riccardo Rigon
 
Hydrological Extremes and Human societies
Hydrological Extremes and Human societies Hydrological Extremes and Human societies
Hydrological Extremes and Human societies
Riccardo Rigon
 
The Science of Water Transport and Floods from Theory to Relevant Application...
The Science of Water Transport and Floods from Theory to Relevant Application...The Science of Water Transport and Floods from Theory to Relevant Application...
The Science of Water Transport and Floods from Theory to Relevant Application...
Riccardo Rigon
 
The Science of Water Transport and Floods from Theory to Relevant Applications
The Science of Water Transport and Floods from Theory to Relevant ApplicationsThe Science of Water Transport and Floods from Theory to Relevant Applications
The Science of Water Transport and Floods from Theory to Relevant Applications
Riccardo Rigon
 
Climaware at the end
Climaware at the endClimaware at the end
Climaware at the end
Riccardo Rigon
 
Hymod model for catchments
Hymod model for catchmentsHymod model for catchments
Hymod model for catchments
Riccardo Rigon
 
Egu2017 pico
Egu2017 picoEgu2017 pico
Egu2017 pico
Riccardo Rigon
 

More from Riccardo Rigon (20)

Models for hazards mapping
Models for hazards mappingModels for hazards mapping
Models for hazards mapping
 
A short introduction to some hydrological extreme phenomena
A short introduction to some hydrological extreme phenomenaA short introduction to some hydrological extreme phenomena
A short introduction to some hydrological extreme phenomena
 
EvaporAzione
EvaporAzioneEvaporAzione
EvaporAzione
 
Francesco Serafin
Francesco Serafin Francesco Serafin
Francesco Serafin
 
Meledrio
MeledrioMeledrio
Meledrio
 
Lisbon talk for SteepStreams
Lisbon talk  for SteepStreamsLisbon talk  for SteepStreams
Lisbon talk for SteepStreams
 
Grids implementation
Grids implementationGrids implementation
Grids implementation
 
Grids
GridsGrids
Grids
 
Some photos from the field
Some photos from the fieldSome photos from the field
Some photos from the field
 
Virtual water fem 07032017
Virtual water fem 07032017Virtual water fem 07032017
Virtual water fem 07032017
 
Dalton Prize Lecture 2017 by Dani Or
Dalton Prize Lecture 2017 by Dani OrDalton Prize Lecture 2017 by Dani Or
Dalton Prize Lecture 2017 by Dani Or
 
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
Projecting Climate Change Impacts on Water Resources in Regions of Complex To...
 
The modern flood forecasting
The modern flood forecastingThe modern flood forecasting
The modern flood forecasting
 
La moderna previsione delle piene
La moderna previsione delle pieneLa moderna previsione delle piene
La moderna previsione delle piene
 
Hydrological Extremes and Human societies
Hydrological Extremes and Human societies Hydrological Extremes and Human societies
Hydrological Extremes and Human societies
 
The Science of Water Transport and Floods from Theory to Relevant Application...
The Science of Water Transport and Floods from Theory to Relevant Application...The Science of Water Transport and Floods from Theory to Relevant Application...
The Science of Water Transport and Floods from Theory to Relevant Application...
 
The Science of Water Transport and Floods from Theory to Relevant Applications
The Science of Water Transport and Floods from Theory to Relevant ApplicationsThe Science of Water Transport and Floods from Theory to Relevant Applications
The Science of Water Transport and Floods from Theory to Relevant Applications
 
Climaware at the end
Climaware at the endClimaware at the end
Climaware at the end
 
Hymod model for catchments
Hymod model for catchmentsHymod model for catchments
Hymod model for catchments
 
Egu2017 pico
Egu2017 picoEgu2017 pico
Egu2017 pico
 

Recently uploaded

(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
Priyanka Aash
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
alexjohnson7307
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
Zilliz
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
BrainSell Technologies
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
SynapseIndia
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
bhumivarma35300
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
David Wilson
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
Arpan Buwa
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
Priyanka Aash
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
sunilverma7884
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
BrainSell Technologies
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
shyamraj55
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
Google Developer Group - Harare
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
ankush9927
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
aslasdfmkhan4750
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
Matthias Neugebauer
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
shanihomely
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
AmandaCheung15
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
softsuave
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
alexjohnson7307
 

Recently uploaded (20)

(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
(CISOPlatform Summit & SACON 2024) Orientation by CISO Platform_ Using CISO P...
 
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
leewayhertz.com-AI agents for healthcare Applications benefits and implementa...
 
Using LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and MilvusUsing LLM Agents with Llama 3, LangGraph and Milvus
Using LLM Agents with Llama 3, LangGraph and Milvus
 
Acumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptxAcumatica vs. Sage Intacct _Construction_July (1).pptx
Acumatica vs. Sage Intacct _Construction_July (1).pptx
 
Tailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer InsightsTailored CRM Software Development for Enhanced Customer Insights
Tailored CRM Software Development for Enhanced Customer Insights
 
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
High Profile Girls call Service Pune 000XX00000 Provide Best And Top Girl Ser...
 
Mastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for SuccessMastering OnlyFans Clone App Development: Key Strategies for Success
Mastering OnlyFans Clone App Development: Key Strategies for Success
 
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and CitiesThe Impact of the Internet of Things (IoT) on Smart Homes and Cities
The Impact of the Internet of Things (IoT) on Smart Homes and Cities
 
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
(CISOPlatform Summit & SACON 2024) Gen AI & Deepfake In Overall Security.pdf
 
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
Girls call Kolkata 👀 XXXXXXXXXXX 👀 Rs.9.5 K Cash Payment With Room Delivery
 
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdfAcumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
Acumatica vs. Sage Intacct vs. NetSuite _ NOW CFO.pdf
 
Integrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecaseIntegrating Kafka with MuleSoft 4 and usecase
Integrating Kafka with MuleSoft 4 and usecase
 
Google I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged SlidesGoogle I/O Extended Harare Merged Slides
Google I/O Extended Harare Merged Slides
 
Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10Computer HARDWARE presenattion by CWD students class 10
Computer HARDWARE presenattion by CWD students class 10
 
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
High Profile Girls Call ServiCe Hyderabad 0000000000 Tanisha Best High Class ...
 
Opencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of MünsterOpencast Summit 2024 — Opencast @ University of Münster
Opencast Summit 2024 — Opencast @ University of Münster
 
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
Premium Girls Call Mumbai 9920725232 Unlimited Short Providing Girls Service ...
 
Zaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdfZaitechno Handheld Raman Spectrometer.pdf
Zaitechno Handheld Raman Spectrometer.pdf
 
Step-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From ScratchStep-By-Step Process to Develop a Mobile App From Scratch
Step-By-Step Process to Develop a Mobile App From Scratch
 
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
leewayhertz.com-Generative AI tech stack Frameworks infrastructure models and...
 

Toolchain for real-time simulations: GSN-MeteoIO-GEOtop

  • 1. Real-Time Toolchain GSN - MeteoIO - GEOtop Thomas Egger Lausanne, 10.12.2009
  • 2. GOAL ● Collect from Sensorscope stations ● Channel data through GSN ● Feed GEOtop with real-time data from GSN ● Publish real-time simulation results
  • 3. Step-by-Step ● Making GEOtop a real-time model ● GEOtop & MeteoIO interaction ● Toolchain (data acquisition & simulation) ● Using GSN Webservice (SOAP/WSDL) ● Demonstration ● Outlook
  • 4. GEOtop Initial Condition ● Strictly linear processing ● No usable start, pause and resume mechanisms ● Difficult configuration of parameters ● Meticulous, time-consuming preparation of data necessary, no safeguards (segfaults) ● How to make a model that can be used reliably with real-time data?
  • 5. GEOtop's Dark Side ● Time required for an expert to run a reference simulation: 4 days ● Extremely complicated internal data structures: tricky even for senior developers ● No checks on parameters files: when format changes -> crashes without any hint of what is wrong ● Moving target: heavy development occurring on GEOtop ➢ requires an expert to run, no automatic, unattended execution possible
  • 6. GEOtop Goals ● Start, pause and resume mechanism ● Use different data sources, without manual preparation (robustness, automatic filtering) ● Easier configuration of simulation ● Increased stability (unattended run)
  • 7. Real-Time GEOtop ● Start, pause and resume through recovery mechanism ● Access to different data sources, data cleaning & filtering through MeteoIO ● Command-Line-Interface for configuration of simulation ● Making code more robust through debugging (segmentation faults, leaking, uninitialized variables)
  • 8. GEOtop Recovery Mechanism ● Save status of simulation after each timestep ● Recovery files in human readable form ● Restore status for a given timestep ● Pitfalls – Global variables (in C: extern AND static vars) – Precision of output fprintf(f, “%.*f”, (int)LDBL_DIG, doublevalue);
  • 9. GEOtop & MeteoIO (1) ● MeteoIO is a library that provides – access to different meteorological data sources through different plugins (meteo data, 2D/3D grids) – advanced methods for data cleaning & filtering as well as spatial interpolation algorithms ● MeteoIO is written in C++, thus wrappers were necessary to “translate” data ● Two MeteoIO plugins developed – GSNIO (accessing GSN data through web service) – GeotopIO (parsing CSV files)
  • 11. Mixing C/C++ ● Using compatible compilers (e.g. gcc) ● Declare linkage for functions (extern keyword) #include <iostream> extern "C" int print(int i, double d) { std::cout << "i = " << i << ", d = " << d; } ● Concept of classes compatible with structs ● Deal with C++ exceptions
  • 12. Toolchain (1) 1.Meteo stations transmit data to Sensorscope servers 2.GSN fetches data by listening to serial data stream, using web service or CSV wrapper 3.GSN triggers a simulation (GEOtop/A3D) if enough data present for next timestep 4.GEOtop/A3D access the data from GSN through the MeteoIO library
  • 13. Toolchain (2) Sensorscope weather stations La Fouly Sensorscope database GSN Network Simulation server Lausanne Lausanne running GEOtop/A3D Lausanne
  • 14. Toolchain (3) Simulation server running GEOtop/A3D Lausanne [3] RSH Sensorscope weather stations La Fouly [4] SOAP [1] GPRS File IO [2] CSV, [5] SFTP Serial, WS Post-processing Sensorscope database GSN Network Lausanne Lausanne
  • 15. What is GSN? ● GSN is a data acquisition network (JAVA) ● It is build around the concept of “sensors” – Real sensors (e.g. thermometer) – Virtual sensors (e.g. averaging filter, GEOtop) ● Many wrappers for different sensors exist
  • 16. Sensorscope GSN Interaction ● For GSN Sensorscope stations are sensors ● CSV Wrapper Importing data by parsing CSV files ● Serial Listener Listening to data packages as they come on a a TCP port (event based messaging) ● Future: Web service or database access
  • 17. Web Services (1) ● By using web services an application can publish its functions on the internet ● Definition of what functions are available in web service description language (WSDL) in XML ● XML used to code and decode data and SOAP to transport it (over HTTP) ● Heterogeneous environments can easily be connected
  • 18. Web Services (2) ● Machine-to-Machine interface (automation) ● 2 ways of development: – bottom-up (start with implementation, generate WSDL) – top-down (start with design, generate skeleton) ● Similar technologies: RMI, CORBA, XML-RPC
  • 19. GSN Web Service ● MeteoIO plugin GSNIO accesses data on GSN servers by means of a web service ● Simple access functions: – getSensors() – getSensorInfo(stationname) – getSensorLocation(stationname) – getMeteoData(station, startDate, endDate) ● GSNWebService.wsdl
  • 20. MeteoIO & GSN Webservice ● wsdl2h generates C++ header file ● soapcpp2 generates C++ stubs ● Incorporated into plugin GSNIO ● Example: Accessing GSN webservice through MeteoIO
  • 21. Demonstration ● Retrieving data through MeteoIO – GEOtopIO plugin – GSNIO plugin ● Script run.pl (Invokes A3D/GEOtop and post-processing) ● GSN webpage displaying results of sensors
  • 22. Outlook ● GSN web interface for starting and running models ● GSN support for reading 2D Grids (DEM) ● Session support (so different users can execute simulation at same time) in both GEOtop and A3D