The document announces the FIWARE Summit to take place in Vienna, Austria from June 12-13, 2023 with the theme "From Data to Value". It discusses smart model right-sizing for alert management in water distribution networks, comparing thin versus fat smart models. The conclusions are to keep models simple, separate development and operations, use a smart data model repository, and focus on user-centricity.
Apidays New York 2024 - The value of a flexible API Management solution for O...
Smart Data Models - Model right-sizing for Alert Management in WDNs Gareth Lewis UNEXE.pptx
1. Vienna, Austria
12-13 June, 2023
#FIWARESummit
From Data
to Value
OPEN SOURCE
OPEN STANDARDS
OPEN COMMUNITY
Smart model right-sizing
for Alert Management in
water distribution
networks
6. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
FIWARE
Output
Service
Context Generation
[input]
Context Consumption
[output]
Context Management
[process]
IoT
Agent
Context
Broker
Output
Service
User
User
Processing
Service
User
Device
Meter
etc
.csv
7. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
Thin vs. Fat Smart Models
Device Model
- Context
- controlledProperty
- Location
Alert Setting
- Min value
- Max value
- observedAt
Alert Status
- Status
- observedAt
Anomaly Setting
- Min value
- Max value
- observedAt
Anomaly Status
- Status
- observedAt
8. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
Thin vs. Fat Smart Models
Device Model
- Context
- controlledProperty
- Location
Alert Setting
- Min value
- Max value
- observedAt
Alert Status
- Status
- observedAt
Anomaly Setting
- Min value
- Max value
- observedAt
Anomaly Status
- Status
- observedAt
9. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
Thin vs. Fat Smart Models
Device Model
- Context
- controlledProperty
- Location
- Alert Setting
- Alert Status
- Anomaly Setting
- Anomaly Status
10. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
Conclusions
▪ KISS
▪ Dev vs. Ops for Devops
▪ Smart data model repository
▪ User-centricity
11. Vienna, 12-13 June, 2023 | #FIWARESummit www.fiware.org
Hosting Partner Keystone Sponsors
Media Partners
Find Us On Stay up to date Be certified and featured
JOIN OUR NEWSLETTER
Hello everyone, welcome to my presentation.
I appreciate this is the last presentation of the session and we’re likely to be short on time, so I will make this suitably brief.
My name is Gareth Lewis and I am a research fellow at the Centre for Water Systems in the University of Exeter in the UK. As there’s a mix of academic and industrial presenters today, I’ll start by saying that one of the benefits of coming at FIWARE-based solutions from an academic perspective is that there’s a lot more scope to be experimental or risky in your approaches and we’ve found this a lot in the FIWARE projects we’ve done, particularly in considering FIWARE from dev and ops perspectives in the traditional devops scheme.
My presentation today will be looking at our approaches to create a workable solution for alert management in water distribution and concerns two projects that were developed in tandem as our ‘first’ FIWARE projects within the centre.
The first is aqua3S, an H2020 funded research project concerned with using FIWARE to develop standardised, secure and safe approaches to digitise water data in European water networks.
-7 pilots across Europe.
The second project is LOTUS, another H2020 funded research project concerned with the development of a low-cost water sensor and associated monitoring and management of water distribution networks in India.
Here’s the data of interest, from the aqua3S project
We have an historic trace of temperature from the broker. The red and purple lines are operator-provided over-topping and under-bottoming alert limits, so this is exactly what you’d expect in traditional management, hard limits.
The green trace is the historic normal data for the device, it’s collected over 5 weeks and then bucketed into a 7-day / weekly slots with 3 data points per day.
From this we generate anomaly limits, the blue trace is a under-bottoming anomaly limit and the black over-topping.
The advantages here are in the space between the anomaly limits and the corresponding alerts, they give us more awareness of impending alerts.
We used FIWARE for both projects, the architecture here showing data collection as context generation, context management as processing and context consumption through the operator webapps.
For device over-topping and underbottoming, it’s as you would expect, data comes in, is processed against the set values from the graph dat and the results are stored in the broker and presented to users on-demand.
Initially, we decided to follow a ‘thin’ model which is common place in OO programming, through the S of SOLID (single responsibility) and through relational database normalisation, i.e. we kept the models as small as possible and had links / relationships between them.
Here we have a device model from the smart data models repo, it contains the context, whatever the controlled property is and some geospatial location data for visualising on a slipmap.
For the alerts and anomalies, we split the data into a pair of models, the setting model contains the settings, so min / max values and the status model contains whether an alert / anomaly has occurred.
This all used observedAt, so the data is stored temporarily and has history.
The massive downside of this approach was all the dependent HTTP requests that it required.
We’d read a device model and then have to collect all the setting and status models for alerts and anomalies, which tended to be quite slow on development PCs and cheap servers we were using.
We also had the issue that because this was development rather than operations, in devops terms, we often had a lot of backlog readings to work though which would require a lot of processing, in comparison with an operational approach where you are generally just collecting a single new reading at any one time.
Needless to say, this approach didn’t work well and we ran into a lot of unfortunate race conditions where the device model controlled property was completely different to the alert / anomaly status , because data was being visualised from the broker before alert and anomaly data was updated.
Instead, we went for a fat model where we put all the attributes / components into a single model and let FIWARE deal with the data coherency.
Here we have all the data that was part of the separate, referenced models, stored in a modified device model
Whilst the model was a lot bigger, it didn’t seem to impact our performance:
All the data could be accessed with a single request
Alert / Anomaly processing was generally a lot simpler, because all the data was in one place
It also removed all the race conditions we had been running into
We also discovered at this point that the users didn’t really care for historic alert/anomaly data, so we could remove the observedAt components, again making the process simpler.
Here are 4 conclusions
Firstly KISS or keep it simple, stupid. In retrospect, I think a lot of our issues with alert management came from trying to implement overly smart solutions with FIWARE when a simple (if less elegant in terms of OO) solution would work perfectly well
Secondly, it’s worth remembering that FIWARE is primarily geared for an operational environment and some of those considerations can impact development activities. From a that perspective, trying to ingest large amounts of data into a broker for testing is not necessarily a good idea.
The smart data model repository is a great resource. Again, it was something I found very odd when I first started working with FIWARE, as a programmer I was used to making my own classes, so having off the shelf models seemed like a strange approach, but there’s a lot of value in being able to see existing approaches, particularly in areas that may be new to you.
Finally, user-centricity. A key issue we ran into with this project was in trying to second guess what users wanted in terms of data and access. This led us to develop an approach that didn’t work and wasn’t what users wanted anyhow.
Thank you.