From idea to production in a day – Leveraging Azure ML and Streamlit to build...
Environmental Risk Assessment Smart Platform
1. A Biological Smart Platform
for the Environmental Risk
Assessment
DAVIDE NARDONE
2. Overview
Components
• Sensors (e.g., biosensors, enviromental sensors, etc.)
• Web-of-Things (WoT) platform
• Web Application
Data Analysis System
• Aggregation, Analysis and Processing of data-stream (online or offline)
• Cloud and Distributed storage systems
• Fuzzy Inference System (FIS)
Goals
• Environmental Risk Assessment (ERA), by using qualitative and quantitative responses
• Visualization and Sharing information among users
3. Web of Things - Multitier Architecture
Things Sensors, Actuators, Devices
Connectivity Communications, Protocols, Networks
Global
Infrastructure
Cloud, Data Center, Software
Application
Web Application, Mobile Application,
Desktop Application
Data Ingestion Big Data, Harvest & Storage of “Thing” data
Data Analysis Soft computing, Mining, Machine Learning
Sensors – temperatures, humidity, water, etc.
Actuators – switch, alarm, power, pressure, etc.
Devices – mobile, tablets, cameras, etc.
Communication – Wi-Fi, LTE, Ethernet, etc.
Protocols – HTTP, MQTT, DDS, etc.
Networks – LAN, WAN
Cloud – Public, Private, Hybrid, Iaas, PaaS, Saas.
Data Center – Google Cloud, Amazon, etc.
Software – IoT Platforms, Dev Kits, APIs, etc.
Streaming, aggregation, data transfer, logging,
monitoring, etc.
Modeling, feature extraction and selection, Fuzzy
Logic, Visualization, Frameworks, etc.
Health Care, Environmental Monitoring,
Environmental Risk Assessment, etc.
4. 1-2. Fog: Things and Connectivity
What is a Biosensors?
• Analytical device which converts a biological
response into an electrical signals
Components
1. Sensitive biological element
2. Transducer or detector element
3. Electronics and signal processors
Standard Communication Protocols:
• WiFi
• Bluetooth
• RFID
• etc.
IoT / WoT Communication Protocols:
• MQTT (Message Queue Telemetry Transport)
• XMPP (Extensible Messaging and Presence Protocol)
• DDS (Data Distribution Service)
• AMQP (Advanced Message Queuing Protocol)
Connectivity
Things
• The information may also come from other kind of sensors
5. 3.Cloud: Global Infrastructure
• Cloud Computing: Model for enabling
ubiquitous, convenient, on-demand network
access to a shared pool of configurable
computing resources (e.g. networks, servers,
storage, applications and services).
Public Hybrid Private
SaaS
Software as a Service
PaaS
Platform as a Service
IaaS
Infrastructure as a Service
LevelofAbstraction
Economies of Scale
FlexibilityofPurpose
Control / Security
• Characteristics:
1. On-demand self service
2. Broad network access
3. Resource pooling
4. Rapid Elasticity
5. Measured services
6. Performace
7. Reduced costs
8. Reliability
9. Multi-tenancy
6. 4.Analytics: Data Ingestion
DEF: A process of obtaining, importing and analyzing data for later use or storage in a database
Apache Kafka
Kafka is a distributed streaming platform useful for:
• Building real-time streaming data pipelines that reliably get data between systems or applications
Apache Spark Streaming
Spark Streaming is an extension of the core Spark API that enables scalable, high-throughput, fault-tolerant stream
processing of live data streams.
Data Sources
Producer
Producer
Producer
Data Ingestion
Topic
◾️◾️◾️◾️◾️◾️
◾️◾️◾️◾️◾️◾️
◾️◾️◾️◾️◾️◾️
◾️
◾️
◾️
Aggregation, Analysis and Processing of stream
Consumer
Consumer
Consumer
◾️
◾️
◾️
Spark
Executors
…
◾️
◾️
◾️
Data Analysis
Mllib
Soft
Computing
Apache Kafka Apache Spark
7. 5.Analytics: Data Analysis
Machine Learning
Fuzzy Inference System
Fuzzifier
Inference
Engine
Defuzzifier
Database Rulebase
Knowledge base
Input Output
• Motivation: Modelling the
complex relationships
between several input
variables.
• Reason: The ERA process
is often performed on
incomplete and imprecise
data.
• Feature extraction: PCA, ICA, etc.
• Feature selection: Sparse and Redudant Representation (Compressed Sensing, SMRS, etc.)
8. 6.Decision: Web Application
The main advantages of WA-based implementation are:
• Cross-platform compatibility: WAs are platform independent, namely, the only requirement is a web
browser.
• Lightweight: WAs require moderate disk space on the client (especially whether Cloud-oriented).
• Integration: WAs integrate easily into other server-side web procedures, such as email and searching.
• No upgrade needed: all new features are implemented on the server and automatically delivered to the
users.
• Stability: users always run the up-to-date WA version and a failure in the WA does not affect the whole
user sytem.
Editor's Notes
Such devices can be used for environmental application e.g., the detection of pesticides and river water contaminants [ref].
MQTT’s goal is to collect data from many devices and transport the data to the IT infrastructure.
The appropriate placement of biosensors depends on their field of application, which may roughly be divided into biotechnology, agriculture, food technology and biomedicine.
RESOURCES
http://www.smalltech.co.uk/?page_id=104
https://sites.google.com/site/mehdidastgheib/biodemetallization
https://learn.sparkfun.com/tutorials/connectivity-of-the-internet-of-things#Thread
https://www.mepits.com/tutorial/180/Biomedical/Wearable-Biosensors
https://www.quora.com/Sensors-What-are-some-of-the-latest-biosensors-available-in-the-market-today
https://med.stanford.edu/news/all-news/2016/01/wearable-device-detects-real-time-changes-in-composition-of-sweat.html
Private/Hybrid Cloud Model: It’s s a cloud-computing services provided to meet the internal needs of the organization/company. The control over the hardware, software, security and other aspect of the data networks are not put in the hands of the third party.
Cloud Computing: Model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications and services).
Private cloud model: This is basically a private network with dedicated servers other network resources.
Data ingested is published by the Kafka broker which streams the data to Kafka consumer process