Presenting the status of the "SOSPilot". A SensorWeb/IoT project where raw AirQuality data from The Netherlands is transformed (ETL) into Web Services like WMS-Time/WFS and in particular the OGC Sensor Observation Service (SOS). Project at Geonovum with partners, a.o.: RIVM, 52 North, and IRCEL - CELINE (Belgium). Work in progress, striving for EIONET IPR and INSPIRE compliance.
Combined tools like Stetl, PostGIS, HeronViewer, GeoServer, and 52North SWE-line. All FOSS. All promising tools as it all just worked, also thanks to the partners' support, within my 100-hour effort. Find more on http://sensors.geonovum.nl
Original version july 2, 2014. Updated with eionet-reporting and Geonovum weather station on okt 27, 2014.
This document contains a list of 1249 earthquake features from the USGS from the past month with magnitudes of 2.5 or greater. It includes information on the time, location, magnitude, and links to details on earthquakes around the world, including several in Oklahoma, Peru, California, Nevada, the Philippines, Puerto Rico, Tennessee, and the U.S. Virgin Islands ranging in magnitude from 2.5 to 4.7.
Test Driven Documentation with Spring Rest Docs JEEConf2017Roman Tsypuk
My talk on JEEConf about Spring Rest Docs, Swagger and Test Driven Documentation.
Material covers:
1. Swagger. SpringFox
2. Swagger approaches and trade-offs
3. Test Driven Development → Test Driven Documentation
4. Live Coding session - how to connect and start using SpringRestDocs
5. Spring Rest Docs features
6. AsciiDoc is awesome (syntax, examples, extensions).
7. Migration of your project from Swagger to SpringRestDocs
Adam Leadbetter is an expert in data management, oceanography, and long-distance running who works for the Marine Institute in Ireland. He is interested in connecting ocean data and emerging technologies to advance oceanography.
The document summarizes FIWARE's IoT platform and the IDAS (IoT Device Management) component. It describes:
1) The typical IoT scenario supported by IDAS, which has been extensively tested and provides developers with measurements from simple sensors.
2) The modular architecture of the new IDAS 4.0, which uses separate IoT agents for different protocols like UL2.0, MQTT, LWM2M etc. This makes installation and extensions easier.
3) The available IoT agents for UL2.0, MQTT, LWM2M, SigFox and how to register devices and send measurements using the UL2.0 agent.
This document proposes OM-JSON, a JSON implementation of the OGC Observations and Measurements (O&M) standard. It provides JSON schemas for representing different types of observations, such as single measurements, time series, geometries, specimens, and collections. Examples are given for each. Issues discussed include how to wrap the encodings for use in APIs/services, differences from other JSON schemas like those from 52North, and potential changes needed to the O&M abstract specification. The motion at the end recommends publishing OM-JSON as an OGC Discussion Paper.
This document contains a list of 1249 earthquake features from the USGS from the past month with magnitudes of 2.5 or greater. It includes information on the time, location, magnitude, and links to details on earthquakes around the world, including several in Oklahoma, Peru, California, Nevada, the Philippines, Puerto Rico, Tennessee, and the U.S. Virgin Islands ranging in magnitude from 2.5 to 4.7.
Test Driven Documentation with Spring Rest Docs JEEConf2017Roman Tsypuk
My talk on JEEConf about Spring Rest Docs, Swagger and Test Driven Documentation.
Material covers:
1. Swagger. SpringFox
2. Swagger approaches and trade-offs
3. Test Driven Development → Test Driven Documentation
4. Live Coding session - how to connect and start using SpringRestDocs
5. Spring Rest Docs features
6. AsciiDoc is awesome (syntax, examples, extensions).
7. Migration of your project from Swagger to SpringRestDocs
Adam Leadbetter is an expert in data management, oceanography, and long-distance running who works for the Marine Institute in Ireland. He is interested in connecting ocean data and emerging technologies to advance oceanography.
The document summarizes FIWARE's IoT platform and the IDAS (IoT Device Management) component. It describes:
1) The typical IoT scenario supported by IDAS, which has been extensively tested and provides developers with measurements from simple sensors.
2) The modular architecture of the new IDAS 4.0, which uses separate IoT agents for different protocols like UL2.0, MQTT, LWM2M etc. This makes installation and extensions easier.
3) The available IoT agents for UL2.0, MQTT, LWM2M, SigFox and how to register devices and send measurements using the UL2.0 agent.
This document proposes OM-JSON, a JSON implementation of the OGC Observations and Measurements (O&M) standard. It provides JSON schemas for representing different types of observations, such as single measurements, time series, geometries, specimens, and collections. Examples are given for each. Issues discussed include how to wrap the encodings for use in APIs/services, differences from other JSON schemas like those from 52North, and potential changes needed to the O&M abstract specification. The motion at the end recommends publishing OM-JSON as an OGC Discussion Paper.
An online training course run by the FIWARE Foundation in conjunction with the i4Trust project. The core part of this virtual training camp (21-24 June 2021) covered all the necessary skills to develop smart solutions powered by FIWARE. It introduces the basis of Digital Twin programming using linked data concepts - JSON-LD and NGSI-LD and combines these with common smart data models for the sharing and augmentation of context data.
In addition, it covers the supplementary FIWARE technologies used to implement the common functions typically required when architecting a complete smart solution: Identity and Access Management (IAM) functions to secure access to digital twin data and functions enabling the interface with IoT and 3rd systems, or the connection with different tools for processing and monitoring current and historical big data.
This 12-hour online training course can be used to obtain a good understanding of FIWARE and NGSI Interfaces and form the basis of studying for the FIWARE expert certification.
Extending this core part, the virtual training camp adds introductory and deep-dive sessions on how FIWARE and iSHARE technologies, brought together under the umbrella of the i4Trust initiative, can be combined to provide the means for the creation of data spaces in which multiple organizations can exchange digital twin data in a trusted and efficient manner, collaborating in the creation of innovative services based on data sharing. In addition, SMEs and Digital Innovation Hubs (DIHs) that go through this complete training and are located in countries eligible under Horizon 2020 will be equipped with the necessary know-how to apply to the recently launched i4Trust Open Call.
The document describes the Schema Editor tool within the OpenIoT platform for managing semantic sensor network schemas and metadata. The Schema Editor allows users to define sensor types, observed properties, and generate RDF instances for new sensor descriptions without requiring expertise in ontologies or RDF syntax. It provides an integrated interface for working with the SSN ontology and extending schemas directly within the OpenIoT platform.
From zero to hero - Easy log centralization with Logstash and ElasticsearchRafał Kuć
Presentation I gave during DevOps Days Warsaw 2014 about combining Elasticsearch, Logstash and Kibana together or use our Logsene solution instead of Elasticsearch.
From Zero to Hero - Centralized Logging with Logstash & ElasticsearchSematext Group, Inc.
Originally presented at DevOpsDays Warsaw 2014. How to set up centralized logging either using ELK stack - Logstash, Elasticsearch, and Kibana or using Logsene.
Context Information Management in IoT enabled smart systems - the basicsFernando Lopez Aguilar
- The Orion Context Broker is used for context information management in IoT systems and allows storing and retrieving context data.
- It uses an entity-attribute model where context information is represented by entities with attributes. Producers can create, update, and retrieve this context data from the broker using RESTful HTTP requests.
- Consumers can subscribe to the broker to receive notifications of attribute updates that match certain criteria. The broker notifies subscribers by invoking callback URLs provided in the subscription.
The document discusses connecting IoT devices to the FIWARE IoT platform. It provides steps to connect a device using the UltraLight 2.0 protocol:
1. Specify connection details for the IoT Agent (IDAS) like IP, ports and API key.
2. Create a device model or use an existing one for the sensor type.
3. Register the device by creating an asset using the REST API and specifying the model.
4. Send observations from the device to IDAS using the UL 2.0 API, including the asset ID and API key.
5. Read the measurements from the Context Broker using NGSI queries to the asset entity.
The document discusses connecting IoT devices to the FIWARE IoT platform. It provides steps to connect a device using the UltraLight 2.0 protocol:
1. Specify connection details for the IoT Agent (IDAS) like IP, ports and API key.
2. Create a device model or use an existing one for the sensor type.
3. Register the device by creating an asset using the REST API and specifying the model.
4. Send observations from the device to IDAS using the UL2.0 API, specifying the asset ID.
5. Read the measurements from IDAS using the NGSI API and specifying the asset ID and attributes.
Monitoring a Kubernetes-backed microservice architecture with PrometheusFabian Reinartz
As many startups of the last decade, SoundCloud’s architecture started as a Ruby-on-Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures.
Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven. The talk will demonstrate the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
This document discusses connecting IoT devices and systems to the FIWARE Lab ecosystem. It provides an overview of the FIWARE IoT proposal and community. It then describes the most common scenarios for connecting IoT providers and consumers to the FIWARE Context Broker using the IDAS/SBC Ultralight 2.0 protocol. Step-by-step instructions are provided for registering models and devices and sending measurements and commands. Additional options for connecting IoT systems like MQTT and LWM2M are also mentioned.
This document summarizes Just van den Broecke's presentation on open sensor networks. It discusses (1) the Smart Emission Nijmegen project from 2015-2017 where citizens measured air quality and noise using low-cost sensors to build an open sensor platform. It then (2) describes the Smart Emission platform resulting from the project which integrates sensor data using open standards and APIs. Finally, it (3) provides an example of bridging IoT sensors to the platform using the SensorThings API to map sensor data from The Things Network to the API in real-time.
Programming IoT Gateways in JavaScript with macchina.ioGünter Obiltschnig
This document discusses programming IoT gateways using JavaScript with the macchina.io toolkit. Macchina.io is an open source modular toolkit for building embedded IoT applications that connect sensors, devices, and cloud services. It uses C++ for performance but includes a JavaScript interface. The document provides an overview of macchina.io and its components, demonstrates how to use JavaScript to access sensors and cloud services, and provides an example of logging sensor data to a database and sending SMS alerts based on sensor readings.
Athens IoT meetup #7 - Create the Internet of your Things - Laurent Ellerbach...Athens IoT Meetup
Laurent Ellerbach, technical evangelist manager at Microsoft, presentation of the "Internet of (his garden) Things", explaining the technical architecture details and decisions.
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It is written in Java and uses a pluggable backend. Presto is fast due to code generation and runtime compilation techniques. It provides a library and framework for building distributed services and fast Java collections. Plugins allow Presto to connect to different data sources like Hive, Cassandra, MongoDB and more.
Docker Logging and analysing with Elastic StackJakub Hajek
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. What aspects should be considered while you design your logging solutions?
Docker Logging and analysing with Elastic Stack - Jakub Hajek PROIDEA
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. We will see the technical presentation on how to manage a large amount of the data in a typical environment with microservices.
This document discusses HTML5 on mobile devices. It begins by explaining why mobile web is growing and why HTML5 is well-suited for mobile. It then provides an overview of what HTML5 is and examples of features like forms, multimedia, geolocation that can be used on mobile. It also discusses considerations for mobile web development like responsive design and frameworks. The document recommends tools for mobile debugging and testing performance.
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsJean-Paul Calbimonte
The document discusses XGSN, an open-source semantic sensing middleware for the Web of Things. It describes how XGSN allows users to store, process, and semantically annotate sensor data from heterogeneous sources. XGSN uses wrappers to integrate different sensor types and formats data using ontologies. It also supports defining virtual sensors that aggregate and correlate multiple data streams. The document considers issues around annotating dynamic sensor data streams at scale and balancing semantic representations with efficient querying and processing of sensor observations.
Service Discovery using etcd, Consul and KubernetesSreenivas Makam
Overview of Service Discovery and Service Discovery using etcd, Consul, Kubernetes and Docker. Presented at Open source meetup, Bangalore(http://www.meetup.com/Bangalore-Open-Source-Meetup/events/229763724/)
Slides originally used in interview by Jonna on De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Aired on oct 1, 2020. Video is on YouTube: https://www.youtube.com/watch?v=3l_a5Up8Rgc
Later adapted/expanded for general online interviews.
THIS IS VERSION 2 AND CURRENT, on SlideShare! Version 1 is still on SlideShare as it is linked...
Some highlights of my professional career.
PLEASE SEE VERSION 2 FOR CURRENT
Slides originally used in interview by Jonna on De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Aired on oct 1, 2020. Video is on YouTube: https://www.youtube.com/watch?v=3l_a5Up8Rgc
Later adapted/expanded for general online interviews.
Some highlights of my professional career.
More Related Content
Similar to SensorWeb SOS Pilot RIVM/Geonovum - Status
An online training course run by the FIWARE Foundation in conjunction with the i4Trust project. The core part of this virtual training camp (21-24 June 2021) covered all the necessary skills to develop smart solutions powered by FIWARE. It introduces the basis of Digital Twin programming using linked data concepts - JSON-LD and NGSI-LD and combines these with common smart data models for the sharing and augmentation of context data.
In addition, it covers the supplementary FIWARE technologies used to implement the common functions typically required when architecting a complete smart solution: Identity and Access Management (IAM) functions to secure access to digital twin data and functions enabling the interface with IoT and 3rd systems, or the connection with different tools for processing and monitoring current and historical big data.
This 12-hour online training course can be used to obtain a good understanding of FIWARE and NGSI Interfaces and form the basis of studying for the FIWARE expert certification.
Extending this core part, the virtual training camp adds introductory and deep-dive sessions on how FIWARE and iSHARE technologies, brought together under the umbrella of the i4Trust initiative, can be combined to provide the means for the creation of data spaces in which multiple organizations can exchange digital twin data in a trusted and efficient manner, collaborating in the creation of innovative services based on data sharing. In addition, SMEs and Digital Innovation Hubs (DIHs) that go through this complete training and are located in countries eligible under Horizon 2020 will be equipped with the necessary know-how to apply to the recently launched i4Trust Open Call.
The document describes the Schema Editor tool within the OpenIoT platform for managing semantic sensor network schemas and metadata. The Schema Editor allows users to define sensor types, observed properties, and generate RDF instances for new sensor descriptions without requiring expertise in ontologies or RDF syntax. It provides an integrated interface for working with the SSN ontology and extending schemas directly within the OpenIoT platform.
From zero to hero - Easy log centralization with Logstash and ElasticsearchRafał Kuć
Presentation I gave during DevOps Days Warsaw 2014 about combining Elasticsearch, Logstash and Kibana together or use our Logsene solution instead of Elasticsearch.
From Zero to Hero - Centralized Logging with Logstash & ElasticsearchSematext Group, Inc.
Originally presented at DevOpsDays Warsaw 2014. How to set up centralized logging either using ELK stack - Logstash, Elasticsearch, and Kibana or using Logsene.
Context Information Management in IoT enabled smart systems - the basicsFernando Lopez Aguilar
- The Orion Context Broker is used for context information management in IoT systems and allows storing and retrieving context data.
- It uses an entity-attribute model where context information is represented by entities with attributes. Producers can create, update, and retrieve this context data from the broker using RESTful HTTP requests.
- Consumers can subscribe to the broker to receive notifications of attribute updates that match certain criteria. The broker notifies subscribers by invoking callback URLs provided in the subscription.
The document discusses connecting IoT devices to the FIWARE IoT platform. It provides steps to connect a device using the UltraLight 2.0 protocol:
1. Specify connection details for the IoT Agent (IDAS) like IP, ports and API key.
2. Create a device model or use an existing one for the sensor type.
3. Register the device by creating an asset using the REST API and specifying the model.
4. Send observations from the device to IDAS using the UL 2.0 API, including the asset ID and API key.
5. Read the measurements from the Context Broker using NGSI queries to the asset entity.
The document discusses connecting IoT devices to the FIWARE IoT platform. It provides steps to connect a device using the UltraLight 2.0 protocol:
1. Specify connection details for the IoT Agent (IDAS) like IP, ports and API key.
2. Create a device model or use an existing one for the sensor type.
3. Register the device by creating an asset using the REST API and specifying the model.
4. Send observations from the device to IDAS using the UL2.0 API, specifying the asset ID.
5. Read the measurements from IDAS using the NGSI API and specifying the asset ID and attributes.
Monitoring a Kubernetes-backed microservice architecture with PrometheusFabian Reinartz
As many startups of the last decade, SoundCloud’s architecture started as a Ruby-on-Rails monolith, which later had to be broken into microservices to cope with the growing size and complexity of the site. The microservices initially ran on an in-house container management and deployment platform. Recently, the company has started to migrate to Kubernetes.
With the introduction of microservices, the existing conventional monitoring setup failed both conceptually and in terms of scalability. Thus, starting in 2012, SoundCloud invested heavily into the development of the open-source monitoring system Prometheus, which was designed for large-scale highly dynamic service-oriented architectures.
Migrating to Kubernetes, it became apparent that Prometheus and Kubernetes are a match made in open-source heaven. The talk will demonstrate the current Prometheus setup at SoundCloud, monitoring a large-scale Kubernetes cluster.
This document discusses connecting IoT devices and systems to the FIWARE Lab ecosystem. It provides an overview of the FIWARE IoT proposal and community. It then describes the most common scenarios for connecting IoT providers and consumers to the FIWARE Context Broker using the IDAS/SBC Ultralight 2.0 protocol. Step-by-step instructions are provided for registering models and devices and sending measurements and commands. Additional options for connecting IoT systems like MQTT and LWM2M are also mentioned.
This document summarizes Just van den Broecke's presentation on open sensor networks. It discusses (1) the Smart Emission Nijmegen project from 2015-2017 where citizens measured air quality and noise using low-cost sensors to build an open sensor platform. It then (2) describes the Smart Emission platform resulting from the project which integrates sensor data using open standards and APIs. Finally, it (3) provides an example of bridging IoT sensors to the platform using the SensorThings API to map sensor data from The Things Network to the API in real-time.
Programming IoT Gateways in JavaScript with macchina.ioGünter Obiltschnig
This document discusses programming IoT gateways using JavaScript with the macchina.io toolkit. Macchina.io is an open source modular toolkit for building embedded IoT applications that connect sensors, devices, and cloud services. It uses C++ for performance but includes a JavaScript interface. The document provides an overview of macchina.io and its components, demonstrates how to use JavaScript to access sensors and cloud services, and provides an example of logging sensor data to a database and sending SMS alerts based on sensor readings.
Athens IoT meetup #7 - Create the Internet of your Things - Laurent Ellerbach...Athens IoT Meetup
Laurent Ellerbach, technical evangelist manager at Microsoft, presentation of the "Internet of (his garden) Things", explaining the technical architecture details and decisions.
Presto is an open source distributed SQL query engine for running interactive analytic queries against data sources of all sizes ranging from gigabytes to petabytes. It is written in Java and uses a pluggable backend. Presto is fast due to code generation and runtime compilation techniques. It provides a library and framework for building distributed services and fast Java collections. Plugins allow Presto to connect to different data sources like Hive, Cassandra, MongoDB and more.
Docker Logging and analysing with Elastic StackJakub Hajek
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. What aspects should be considered while you design your logging solutions?
Docker Logging and analysing with Elastic Stack - Jakub Hajek PROIDEA
Collecting logs from the entire stateless environment is challenging parts of the application lifecycle. Correlating business logs with operating system metrics to provide insights is a crucial part of the entire organization. We will see the technical presentation on how to manage a large amount of the data in a typical environment with microservices.
This document discusses HTML5 on mobile devices. It begins by explaining why mobile web is growing and why HTML5 is well-suited for mobile. It then provides an overview of what HTML5 is and examples of features like forms, multimedia, geolocation that can be used on mobile. It also discusses considerations for mobile web development like responsive design and frameworks. The document recommends tools for mobile debugging and testing performance.
XGSN: An Open-source Semantic Sensing Middleware for the Web of ThingsJean-Paul Calbimonte
The document discusses XGSN, an open-source semantic sensing middleware for the Web of Things. It describes how XGSN allows users to store, process, and semantically annotate sensor data from heterogeneous sources. XGSN uses wrappers to integrate different sensor types and formats data using ontologies. It also supports defining virtual sensors that aggregate and correlate multiple data streams. The document considers issues around annotating dynamic sensor data streams at scale and balancing semantic representations with efficient querying and processing of sensor observations.
Service Discovery using etcd, Consul and KubernetesSreenivas Makam
Overview of Service Discovery and Service Discovery using etcd, Consul, Kubernetes and Docker. Presented at Open source meetup, Bangalore(http://www.meetup.com/Bangalore-Open-Source-Meetup/events/229763724/)
Similar to SensorWeb SOS Pilot RIVM/Geonovum - Status (20)
Slides originally used in interview by Jonna on De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Aired on oct 1, 2020. Video is on YouTube: https://www.youtube.com/watch?v=3l_a5Up8Rgc
Later adapted/expanded for general online interviews.
THIS IS VERSION 2 AND CURRENT, on SlideShare! Version 1 is still on SlideShare as it is linked...
Some highlights of my professional career.
PLEASE SEE VERSION 2 FOR CURRENT
Slides originally used in interview by Jonna on De Grote Geo Show, a live-streaming webshow by OSGeo.nl. Aired on oct 1, 2020. Video is on YouTube: https://www.youtube.com/watch?v=3l_a5Up8Rgc
Later adapted/expanded for general online interviews.
Some highlights of my professional career.
Presentation at LoRaWAN TheThingsNetwork makers event (make a Pet-Tracker with LoRa) organized by Ruimteschepper at Warehouse of Innovation Eindhoven (also home to IoT Eindhoven Meetup Group). Summarizes Smart Emission architecture, thoughts on tech for Spatio-Temporal Data, Internet of Things sorry Internet of Silos. Ending with a demo on LoRaWAN TheThingsNetwork integration with the OGC SensorThings API using MQTT Bridging.
Presentation provided at "Geo Gebruikersfestival", okt 31, 2018, Amersfoort, The Netherlands. Sketches how from several Sensor projects the Smart Emission Platform emerged and migrated to the PDOK Platform. How a next step could be be a Dutch national Sensor SDI asa a federated/distributed architecture. Special attention is given to APIs, in particular the SensorThings API.
Opening by me as Chair OSGeo.nl Foundation, the Dutch Local Chapter of OSGeo.org at FOSS4GNL Conference on July 11, 2018 at Aeres Highschool, Almere, the Netherlands.
This document provides an overview of Stetl, an open source spatial ETL tool. Stetl uses Python components and configuration files to orchestrate multi-step ETL workflows. Key concepts discussed include using Stetl to extract data from various sources like GML, CSV and ZIP files, transform it using filters like XSLT, and load it into targets like PostGIS, GeoPackage and WFS. Example Stetl configurations are provided to extract Dutch cadastre data from GML ZIP files and load it to PostGIS and GeoPackage. Stetl is also used in the Smart Emission platform to harvest sensor data from multiple sources and perform validation, calibration and aggregation before publishing to web services.
Opening words by Just van den Broecke - Chair of OSGeo.nl Foundation, the Dutch local Chapter of OSGeo.org on joint OSGeo.nl OpenStreetMap NL NJ Party in cafe Dudok Hilversum. Review of OSGeo.nl Events in 2017 and upcoming events in 2018, also international events.
Slides used at the opening of the OSGeo.nl Day on nov 22, 2017 at GeoBuzz in Den Bosch, The Netherlands. Note the new designs of OSGeo localized for OSGeo.nl.
Presented together with Jan-Willem van Aalst and Frank Steggink at the Dutch CartoDay, March 15, 2017. See full program here: http://www.cartodag.nl/programma-cartodag-2017. Subject was how to eventually create online topographical maps from Open but Raw datasets: how to transform (ETL), create maps (QGIS), and publish on the web using Open Source service components like MapServer and MapProxy via http://map5.nl
Abstract in Dutch:
Om bruikbare kaarten te maken is tegenwoordig veel goede open geodata beschikbaar. Maar wat komt er allemaal bij kijken om dat zo praktisch mogelijk te doen? Deze sessie neemt je mee in de levenscyclus van open geodata met tooling zoals NLExtract , GDAL en QGIS . Een greep uit wat er langs komt in dit uur: het maken van PostGIS databases uit PDOK data; het maken van hillshading (reliëfschaduw) uit de AHN2/3; het maken van de OpenTopo kaartbeelden uit combinaties van PDOK bestanden; en het tegelen en opnieuw publiceren van allerlei kaartproducten, bijvoorbeeld op map5.nl .
5 minute pitch held at GeoBuzz 2016 Den Bosch for the OGT Award (which we received in Category "Developers"!). It introduces NLExtract, a toolset to convert Dutch National Open Geodata sets to manageable formats such as PostGIS.
The document discusses the Smart Emission project, which aims to deploy low-cost air quality sensors throughout cities to create inclusive citizen sensor networks for environmental monitoring. It outlines key questions around sensor selection, calibration, spatial deployment, data management, and citizen involvement. The goals are to make pollution monitoring more transparent and democratic through open data, while using cost-effective sensing devices and geospatial approaches to engage citizens in analysis and help work towards more sustainable cities.
Explains basics of Stetl, an Open Source spatial ETL framework. More on http://stetl.org. Talk given on GeoPython conference, June 24, 2016, Basel, Switserland.
High-level presentation on the Data-infastructure for project-team and participating citizens at Radboud Univeristy Nijmegen, may 26, 2016 for the Smart Emission project: "Het Smart Emission project draait om het in kaart brengen van luchtkwaliteit, geluid, trillingen en meteorologische indicatoren in de stad op een fijnmazig schaalniveau, door inwoners met zogenoemde burger-sensor-netwerken." (Bron: http://smartemission.nl). Project partners: Radboud Universiteit, Gemeente Nijmegen, Rijksinstituut voor Volksgezondheid en Milieu (RIVM), Geonovum, Intemo, CityGIS and offcourse Nijmegen citizens.
Presentation given at the OSGeo.nl Day http://osgeo.nl at GeoBuzz Conference on nov 25, 2015. Overview of NLExtract and in particular ETL (conversion) for BAG, the open geo-dataset for Dutch Addresses and Buildings.
Slides I presented at the Dutch "Doorbraak 3D - Trekkersoverleg" ( 3D Breakthrough Representatives Meetup) at Geonovum, sept 3, 2015. Trying to provide an overview of 3D Open Standards efforts and suggestions for future actions/pilots with some of the CesiumJS-based tiling standards using Dutch public open base registry data ("Basis Registraties").
Eng: originally planned as a talk/workshop on hiking with GPS, but evolving into a wider talk about the evolution of navigation and Natural Navigation techniques.
See also the main website: http://gps.justobjects.nl.
Below Dutch:
Plaatsbepaling en navigatie is iets van alle tijden. Zij is mee-geëvolueerd met de mensheid. Vandaag de dag doen we navigatie met GPS via bijv “De TomTom” of steeds vaker via onze Smartphone. We dreigen daarmee echter belangrijke vaardigheden te verliezen zoals kaartlezen en het gebruik van kompas. Maar deze laatste twee middelen waren ook weer een stap in de lange evolutie van Navigatie. Zoals de meesten van jullie weten heb ik veel interesse in de evolutie van de mensheid. In die context heb ik de laatste tijd nagedacht hoe navigatie zich heeft ontwikkeld. Ik zal dit praatje chronologisch opbouwen. Dan komen we vanzelf uit bij "Wandelen met de GPS" waar de nadruk op gaat liggen deze middag.
Highlights and achievements in 2014 from the OSGeo.nl, the Dutch-speaking Local Chapter of the Open Source Geospatial Foundation, OSGeo.org. Presented jan 11 2015 by Gert-Jan and Just at the OSGeo.nl/OpenStreetMap NL new years party. See more presentations from that meeting on our website: http://osgeo.nl
My talk on the OSGeo.nl Day 2014 on nov 25, 2014, Den Bosch. This event was organized by the OSGeo.nl, the Dutch Language Chapter of OSGeo.org and embedded in the GeoBuzz conference. Main subject is how to transform and visualize (unlock) Dutch Open Geo-datasets using open source tools like NLExtract (http://nlextract.nl), Stetl (http://www.stetl.org). In particular, there is a last minute addition on unlocking 3D data, Top10NL3D, using CesiumJS (http://www.cesiumjs.org) , an Open Source browser/WebGL-based 3D visualization framework ala Google Earth (but open and without plugins). The 3D part of this talk was also given the same day at the "3D Doorbraak" workshop (Jantien Stoter et al).
Big Data - Introduction and Research Topics - for Dutch KadasterJust van den Broecke
This document discusses big data and big geo data. It begins by defining big data and noting the challenges of collecting, managing, storing, searching, sharing, transferring, analyzing and visualizing large, complex datasets. It then discusses using big data as an opportunity rather than a threat by developing new ways to store and analyze diverse data to generate insights. Specific examples discussed include data used by Amazon, Obama's 2012 re-election campaign, meteorology, lidar data, and linked open data projects in the UK and Netherlands. The document raises potential applications of big data for the Dutch Cadastre, including data analysis, linking datasets, real-time access, multidimensional visualization and querying basic registrations.
UI5con 2024 - Keynote: Latest News about UI5 and it’s EcosystemPeter Muessig
Learn about the latest innovations in and around OpenUI5/SAPUI5: UI5 Tooling, UI5 linter, UI5 Web Components, Web Components Integration, UI5 2.x, UI5 GenAI.
Recording:
https://www.youtube.com/live/MSdGLG2zLy8?si=INxBHTqkwHhxV5Ta&t=0
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
Most important New features of Oracle 23c for DBAs and Developers. You can get more idea from my youtube channel video from https://youtu.be/XvL5WtaC20A
Top Benefits of Using Salesforce Healthcare CRM for Patient Management.pdfVALiNTRY360
Salesforce Healthcare CRM, implemented by VALiNTRY360, revolutionizes patient management by enhancing patient engagement, streamlining administrative processes, and improving care coordination. Its advanced analytics, robust security, and seamless integration with telehealth services ensure that healthcare providers can deliver personalized, efficient, and secure patient care. By automating routine tasks and providing actionable insights, Salesforce Healthcare CRM enables healthcare providers to focus on delivering high-quality care, leading to better patient outcomes and higher satisfaction. VALiNTRY360's expertise ensures a tailored solution that meets the unique needs of any healthcare practice, from small clinics to large hospital systems.
For more info visit us https://valintry360.com/solutions/health-life-sciences
What to do when you have a perfect model for your software but you are constrained by an imperfect business model?
This talk explores the challenges of bringing modelling rigour to the business and strategy levels, and talking to your non-technical counterparts in the process.
UI5con 2024 - Bring Your Own Design SystemPeter Muessig
How do you combine the OpenUI5/SAPUI5 programming model with a design system that makes its controls available as Web Components? Since OpenUI5/SAPUI5 1.120, the framework supports the integration of any Web Components. This makes it possible, for example, to natively embed own Web Components of your design system which are created with Stencil. The integration embeds the Web Components in a way that they can be used naturally in XMLViews, like with standard UI5 controls, and can be bound with data binding. Learn how you can also make use of the Web Components base class in OpenUI5/SAPUI5 to also integrate your Web Components and get inspired by the solution to generate a custom UI5 library providing the Web Components control wrappers for the native ones.
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...XfilesPro
Wondering how X-Sign gained popularity in a quick time span? This eSign functionality of XfilesPro DocuPrime has many advancements to offer for Salesforce users. Explore them now!
14 th Edition of International conference on computer visionShulagnaSarkar2
About the event
14th Edition of International conference on computer vision
Computer conferences organized by ScienceFather group. ScienceFather takes the privilege to invite speakers participants students delegates and exhibitors from across the globe to its International Conference on computer conferences to be held in the Various Beautiful cites of the world. computer conferences are a discussion of common Inventions-related issues and additionally trade information share proof thoughts and insight into advanced developments in the science inventions service system. New technology may create many materials and devices with a vast range of applications such as in Science medicine electronics biomaterials energy production and consumer products.
Nomination are Open!! Don't Miss it
Visit: computer.scifat.com
Award Nomination: https://x-i.me/ishnom
Conference Submission: https://x-i.me/anicon
For Enquiry: Computer@scifat.com
Mobile App Development Company In Noida | Drona InfotechDrona Infotech
Drona Infotech is a premier mobile app development company in Noida, providing cutting-edge solutions for businesses.
Visit Us For : https://www.dronainfotech.com/mobile-application-development/
Consistent toolbox talks are critical for maintaining workplace safety, as they provide regular opportunities to address specific hazards and reinforce safe practices.
These brief, focused sessions ensure that safety is a continual conversation rather than a one-time event, which helps keep safety protocols fresh in employees' minds. Studies have shown that shorter, more frequent training sessions are more effective for retention and behavior change compared to longer, infrequent sessions.
Engaging workers regularly, toolbox talks promote a culture of safety, empower employees to voice concerns, and ultimately reduce the likelihood of accidents and injuries on site.
The traditional method of conducting safety talks with paper documents and lengthy meetings is not only time-consuming but also less effective. Manual tracking of attendance and compliance is prone to errors and inconsistencies, leading to gaps in safety communication and potential non-compliance with OSHA regulations. Switching to a digital solution like Safelyio offers significant advantages.
Safelyio automates the delivery and documentation of safety talks, ensuring consistency and accessibility. The microlearning approach breaks down complex safety protocols into manageable, bite-sized pieces, making it easier for employees to absorb and retain information.
This method minimizes disruptions to work schedules, eliminates the hassle of paperwork, and ensures that all safety communications are tracked and recorded accurately. Ultimately, using a digital platform like Safelyio enhances engagement, compliance, and overall safety performance on site. https://safelyio.com/
Using Query Store in Azure PostgreSQL to Understand Query PerformanceGrant Fritchey
Microsoft has added an excellent new extension in PostgreSQL on their Azure Platform. This session, presented at Posette 2024, covers what Query Store is and the types of information you can get out of it.
7. Scope
1. Intro
From: ”Building bridges: experiences and lessons learned from the implementation of INSPIRE and e-reporting of air quality data in Europe"
by: Alexander Kotsev, Olav Peeters, Paul Smits and Michel Grothe. - Earth Sci Inform DOI 10.1007/s12145-014-0160-8
SOS
WMS
WFS
Data
14. 2. Data Transformation (ETL)
Multistep ETL -"
Advantages
• backup of source XML data files"
• incrementally build up of history past the last month"
• in case of (design) errors we can always restart anew"
• simpler ETL scripts than “all-in-one” (divide & conquer)"
• migration with changed in 52N SOS DB schema simpler"
• prepared for IPR/INSPIRE ETL "
• OWS server (WMS/WFS evt WCS) can directly use Core AQ DB"
• many possibilities with VIEWs on Core AQ DB"
• aggregations"
• last measured values"
• peak values"
• Voronoi polygons
15. 2. Data Transformation (ETL)
ETL Step 1"
File Harvester
Raw XML Files
“Blobs” + meta
in PostGIS
16. 2. Data Transformation (ETL)
ETL Step 2"
Raw AQ Measurements
“Blobs” + meta
in PostGIS
Records in
PostGIS
17. 2. Data Transformation (ETL)
ETL Step 2"
Stations
Edits + GDAL ogr2ogr
Stations CSV
EIONET
Records in
PostGIS
18. 2. Data Transformation (ETL)
ETL Step 3"
SOS Publication
Via SOS-Transaction Protocol
Raw Records
PostGIS
S
O
S
-
T 52N SOS
PostGIS DB
21. 2. Data Transformation (ETL)
ETL Tech"
Progress Tracking
Track
Last Processed
Record in DB
22. 2. Data Transformation (ETL)
ETL Tech"
Tool: Stetl
• Framework for Streaming ETL"
• Proven for Dutch datasets (Top10NL and BGT)"
• Proven for INSPIRE ETL (BAG to AD, Top10 to TN)"
• Integration with native libs: libXML, libXSLT, GDAL/OGR"
• Declarative programming using config files"
• Call-back programming model"
• Extensibility via own custom modules"
• Free and Open Source (GPL) in Python"
"
More on stetl.org and presentation
26. 3. Services
Services"
WMS/WFS
• On RAW AQ Data tables"
• Prepare data via Postgres VIEWs"
• Using GeoServer"
• Using WMS-Time"
"
Service URL "
sensors.geonovum.nl/gs/ows
Viewer
sensors.geonovum.nl/heronviewer
27. CREATE VIEW rivm_lml.measurements_stations AS!
SELECT m.gid, m.station_id, s.municipality, m.component, m.sample_time,
m.sample_value, s.point, m.validated,!
m.file_name, m.insert_time, m.sample_id,!
s.local_id, s.eu_station_code, s.altitude, s.area_classification,!
s.activity_begin, s.activity_end!
FROM rivm_lml.measurements as m!
INNER JOIN rivm_lml.stations as s ON m.station_id =
s.natl_station_code;
Services "
WMS/WFS via VIEWs
3. Services
Measurements with Station info, a.o. GeoLocation
28. Services "
WMS/WFS via VIEWs
3. Services
Measurements with Station info, a.o. GeoLocation
29. -- per component last-captured measurements!
CREATE VIEW rivm_lml.v_last_measurements_NO2 AS!
SELECT DISTINCT ON (station_id) station_id,!
municipality, sample_time, sample_value, point, validated, gid, sample_id!
FROM rivm_lml.measurements_stations WHERE component = 'NO2' ORDER BY
station_id, gid DESC;!
"
CREATE VIEW rivm_lml.v_last_measurements_O3 AS!
SELECT DISTINCT ON (station_id) station_id,!
municipality, sample_time, sample_value, point, validated, gid, sample_id!
FROM rivm_lml.measurements_stations WHERE component = 'O3' ORDER BY
station_id, gid DESC;!
"
.!
.!
"
-- per component time-series measurements!
CREATE VIEW rivm_lml.v_measurements_NO2 AS!
SELECT station_id,!
municipality, sample_time, sample_value, point, validated, gid, sample_id!
FROM rivm_lml.measurements_stations WHERE component = 'NO2';!
Services "
WMS/WFS via VIEWs
3. Services
32. 3. Services
Services"
SOS
• Using 52North SOS"
• Easy setup via Tomcat deploy + Admin GUI"
• Data population via SOS-T"
"
Service URL "
http://sensors.geonovum.nl/sos/service?
40. AQ Reporting - Architecture
4. Clients
AQ
ETL
EU
Database Database
Web Services
JSON Docs
Dataflows
B,C,D,E,F
WFS SOS Custom
41. AQ Reporting - ETL
4. Clients
Stetl
Dataflows
B,C,D,E,F
XML Docs
XML Template
Docs (Jinja2)
Custom
Python
Stetl
ETL Config
Web Services
WFS SOS Custom
42. # Example dataflow-D (Assessment Methods metadata) report generation using Stetl with Jinja2 templating filter.
# The input data comes directly from a WFS providing station info.!
# The template files and global variables (globals.json) are under templates folder.!
#!
# All rendered output under output/!
#!
[etl]!
chains = input_json|dataflow-D_template | output_dataflow-D_file!
"
[input_json]!
class = inputs.fileinput.JsonFileInput!
file_path = http://sensors.geonovum.nl/gs/ows?service=WFS&request=GetFeature
&typeName=sensors:active_stations&outputformat=JSON&srsName=EPSG:4326!
"
# Advanced gml templating with globals for more or less static content!
# in rivm-globals.json like id-prefixes and contact info etc!
[dataflow-D_template]!
class = filters.templatingfilter.Jinja2TemplatingFilter!
template_file = templates/dataflow-D.jinja2!
template_globals_path = input/globals-rivm.json!
"
[output_dataflow-D_file]!
class = outputs.fileoutput.FileOutput!
file_path = output/dataflow-D.xml!
Stetl Config - Dataflow D
4. Clients
ETL Chain
WFS Input
XML
Template
Output
File
45. AQ Reporting Status
• Dataflows B,C,D more or less done
• Main issue is lack of source data
• See GitHub for code:
https://github.com/Geonovum/sospilot/tree/
master/src/aq-report
• Very quick way to develop, about one day per
dataflow
46. AQ Reporting
Recommendations
• Store all relevant data in database(s) - PostgreSQL!
• Expose data via web services:WFS, SOS, Custom
• Let these web services provide JSON (i.s.o. XML)
• Keep the SOS basic/let ETL do the AQ reporting
• Use template substitution as ETL mechanism
58. Open Items/TODO
6. Discussion
• Source data"
• http://www.lml.rivm.nl/xml or"
• http://www.lml.rivm.nl/sos "
• Stations info "
• name attribute Eionet only municipality"
• Standardize/Harmonize SOS Model Elements"
• pollutant naming, codelists"
• Time format (timezone) from source to SOS"
• INSPIRE-related"
• 2 Client-frameworks: SOS.js and SWC"
• (Historical) weather data in WMS-T and SOS"
"
Tracked on https://github.com/Geonovum/sospilot/issues