SlideShare a Scribd company logo
1
Lecture 6: IoT Data Processing
EEEM048/COM3023- Internet of Things
Prof. Payam Barnaghi
Centre for Vision, Speech and Signal Processing (CVSSP)
Electrical and Electronic Engineering Department
University of Surrey
Autumn 2018
Wireless Sensor (and Actuator)
Networks
Sink
node Gateway
Core network
e.g. InternetGateway
End-user
Computer services
- The networks typically run Low Power Devices
- Consist of one or more sensors, could be different type of sensors (or actuators)
Operating
Systems?
Services?
Protocols?
Protocols?
In-node Data
Processing
Data
Aggregation/
Fusion
Inference/
Processing of
IoT data
3
Key characteristics of IoT devices
−Often inexpensive sensors (actuators) equipped with a radio transceiver for
various applications, typically low data rate ~ 10-250 kbps (but not always-
e.g. CCTV and video/sound data will require higher bandwidth/data rates).
−Deployed in large numbers
−The sensors may need to coordinate to perform a desired task.
−The acquired information (periodic or event-based) is reported back to the
information processing centre (in some cases in-network processing is
required) or the information/alert is communicated to another device
(M2M communication/interaction)
−Solutions are often application-dependent.
3
4
Beyond conventional sensors
−Human as a sensor (citizen sensors)
−e.g. tweeting real world data and/or events
−Software sensors
−e.g. Software agents/services generating/representing data
Road block, A3
Road block, A3
5
The benefits of data processing in IoT
− Convert (billions of) smart meter readings to better predict and balance
power consumption.
− Analyze thousands of traffic, pollution, weather, congestion, public
transport and event sensory data to provide better traffic and smart city
management.
− Monitor patients, in-home care and several other healthcare
applications…
− The applications require: real-time, reliable, efficient (for low power and
resource limited nodes), and scalable solutions.
Adapted from: What is Bog Data?, IBM
6
IoT Data Access
− Publish/Subscribe (long-term/short-term)
− Ad-hoc query
− The typical types of data request for sensory data:
− Query based on
− ID (resource/service) – for known resources; eg. data from device ABC001?
− Location e.g. traffic data from junction ABC01
− Type e.g. all the temperature readings
− Time – requests for fresh or historical data; e.g. all the collected data at 9:00am
− One of the above + a range [+ unit of measurement]
− Type/Location/Time + A combination of Quality of Information attributes
− An entity of interest (a feature of an entity on interest)- e.g. temperature of a room;
quality of air in an area;
− Complex DataTypes (e.g. pollution data could be a combination of different types)
Sensor Data
− The sensory data represents physical world observations and
measurements and requires time and location and other
descriptive attributes to make the data more meaningful.
− For example, a temperature value of 15 degree will be more
meaningful when it is described with spatial (e.g. Guildford
town centre) and temporal (e.g. 8:15AM GMT, 07-11-2018),
and unit (e.g. Celsius) attributes.
− The sensory data can also include other detailed meta-data
that describe quality or device related attributes (e.g.
Precision, Accuracy).
7
Sensor Data
15, C, 08:15, 51.243057, -0.589444
8
Data Processing and Interpretation
−Processing and Interpretation of data (this
week)
−Meta-data enhancement, annotation and
semantically described IoT data
(next week)
9
IoT Data Processing
WSN
WSN
WSN
WSN
WSN
Network-enabled
Devices
Network-enabled
Devices
Network
services/storage
and processing
units Data/service access
at application level
Data collections and
processing within
the networks
Data Discovery
Service/
Resource
Discovery
11
In-network processing
− Mobile Ad-hoc Networks can be seen as a set of nodes that deliver bits
from one end to the other;
− WSNs, on the other end, are expected to provide information, not
necessarily original bits
− Gives additional options
− e.g., manipulate or process the data in the network
− Main example: aggregation
− Applying aggregation functions to a obtain an average value of measurement
data
− Typical functions: minimum, maximum, average, sum, …
− Not amenable functions: median
Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
12
In-network processing
− Depending on application, more sophisticated processing of data can take
place within the network
− Example edge detection: locally exchange raw data with neighboring nodes,
compute edges, only communicate edge description to far away data sinks.
− Example tracking/angle detection of signal source: conceive of sensor nodes as
a distributed microphone array, use it to compute the angle of a single source,
only communicate this angle, not all the raw data.
− Use temporal and spatial correlations
− Observed signals might vary only slowly in time; so no need to transmit all
data at full rate all the time.
− Signals of neighboring nodes are often quite similar; only try to transmit
differences.
Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
13
Data-centric networking
− In typical networks (including ad-hoc networks), network transactions are
addressed to the identities of specific nodes
− A “node-centric” or “address-centric” networking paradigm
− In a redundantly deployed sensor network, specific source of an event,
alarm, etc. might not be important
− Redundancy: e.g., several nodes can observe the same area
− Thus: focus networking transactions on the data directly instead of their
senders and transmitters; i.e. data-centric networking
− Specially this idea is reinforced by the fact that we might have multiple sources
to provide information and observations from the same or similar areas.
Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks
Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
Data Aggregation
− Computing a smaller representation of a number of data items (or
messages) that is extracted from all the individual data items.
− For example computing min/max or mean of sensor data.
− More advance aggregation solutions could use approximation techniques
to transform high-dimensionality data to lower-dimensionality
abstractions/representations.
− The aggregated data can be smaller in size, represent patterns/abstractions;
so in multi-hop networks, nodes can receive data form other node and
aggregate them before forwarding them to a sink or gateway.
− The aggregation can happen on a sink/gateway node.
Aggregation example
− Reduce number of transmitted bits/packets by applying an aggregation
function in the network
1
1
3
1
1
6
1
1
1
1
1
1
Source: Holger Karl, Andreas Willig, Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks, chapter 3, Wiley, 2005 .
Efficacy of an aggregation mechanism
− Accuracy: difference between the resulting value or representation and
the original data
− Some solutions can be lossless or lossy depending on the applied
techniques.
− Completeness: the percentage of all the data items that are included in the
computation of the aggregated data.
− Latency: delay time to compute and report the aggregated data
− Computation foot-print; complexity;
− Overhead: the main advantage of the aggregation is reducing the size of
the data representation;
− Aggregation functions can trade-off between accuracy, latency and overhead;
− Aggregation usually happens closer to the source.
Publish/Subscribe
− Achieved by publish/subscribe paradigm
− Idea: Entities can publish data under certain names
− Entities can subscribe to updates of such named data
− Conceptually: Implemented by a software bus
− Software bus stores subscriptions, published data; names used as filters;
subscribers notified when values of named data changes
Software bus
Publisher 1 Publisher 2
Subscriber 1 Subscriber 2 Subscriber 3
Source: Holger Karl, Andreas Willig, Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks, chapter 12, Wiley, 2005 .
Message Brokers
− For example, RabbitMQ is one of the widely deployed open
source message brokers.
− Protocols such as AMQP, MQTT.
− A producer is a user application that sends messages.
− A queue is a buffer that stores messages.
− A consumer is a user application that receives messages.
18
Source: https://www.rabbitmq.com/tutorials/tutorial-three-python.html
MQTT Pub/Sub Protocol
− MQTelemetry Transport (MQTT) is a lightweight broker-based
publish/subscribe messaging protocol.
− MQTT is designed to be open, simple, lightweight and easy to implement.
− These characteristics make MQTT ideal for use in constrained
environments, for example in IoT.
−Where the network is expensive, has low bandwidth or is
unreliable
−When run on an embedded device with limited processor or
memory resources;
− A small transport overhead (the fixed-length header is just 2 bytes), and
protocol exchanges minimised to reduce network traffic
Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html
MQTT Messages
− The MQTT packet or message format consists of a 2 byte
fixed header (always present) + Variable-header (not
always present)+ payload (not always present)*.
− It supports publish/subscribe message pattern to provide one-
to-many message distribution and decoupling of applications
− A messaging transport that is agnostic to the content of the
payload
− The use of TCP/IP to provide basic network connectivity
20* Source: http://www.steves-internet-guide.com/mqtt-protocol-messages-overview/
MQTT- Quality of Service
− Three qualities of service for message delivery:
− "At most once", where messages are delivered according to the best
efforts of the underlying TCP/IP network. Message loss or duplication
can occur.
−This level could be used, for example, with ambient sensor data
where it does not matter if an individual reading is lost as the next
one will be published soon after.
− "At least once", where messages are assured to arrive but duplicates
may occur.
− "Exactly once", where message are assured to arrive exactly once.This
level could be used, for example, with billing systems where duplicate
or lost messages could lead to incorrect charges being applied.
Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html
MQTT Message Format
− The message header for each MQTT command message contains a fixed header.
− Some messages also require a variable header and a payload.
− The format for each part of the message header:
Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html
— DUP: Duplicate delivery
— QoS: Quality of Service
— RETAIN: RETAIN flag
—This flag is only used on PUBLISH messages. When a client sends a PUBLISH
to a server, if the Retain flag is set (1), the server should hold on to the message
after it has been delivered to the current subscribers.
—This allows new subscribers to instantly receive data with the retained or
Last Known Good value.
Sensor Data as time-series data
− The sensor data (or IoT data in general) can be seen as time-
series data.
− A sensor stream refers to a source that provide sensor data
over time.
− The data can be sampled/collected at a rate (can be also
variable) and is sent as a series of values.
− Over time, there will be a large number of data items
collected.
− Using time-series processing techniques can help to reduce
the size of the data that is communicated;
−Let’s remember, communication can consume more
energy than computaiton;
Time series data and segmentation
− The collection of several points spaced in time, having a
temporal order is known as time series data.Time series can
be analysed using various techniques such as clustering,
classification, regression and prediction in the fields of data
mining, machine learning, signal processing, communication
engineering, and statistics.
− Given a time series T containing n data points, segmentation is
defined as the construction of a model T’, from l piecewise
segments (l < n) such that T’ closely approximates T.
24
Time series data
25
Segment 1 Segment 2 Segment …
Sensor Data as time-series data
− Different representation method that introduced for time-series data can
be applied.
− The goal is to reduce the dimensionality (and size) of the data, to find
patterns, detect anomalies, to query similar data;
− Dimensionality reduction techniques transform a data series with n items
to a representation with w items where w < n.
− This functions are often lossy in comparison with solutions like normal
compression that preserve all the data.
− One of these techniques is called Symbolic Aggregation Approximation
(SAX).
− SAX was originally proposed for symbolic representation of time-series
data; it can be also used for symbolic representation of time-series sensor
measurements.
− The computational foot-print of SAX is low; so it can be also used as a an
in-network processing technique.
27
In-network processing
Using Symbolic Aggregate Approximation (SAX)
SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols
over the original sensor time-series data (green)
Source: P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data", in Proc. of the IEEE Sensors 2012, Oct. 2012.
fggff
jfhig
fggff
Symbolic Aggregate Approximation
(SAX)
− SAX transforms time-series data into symbolic string representations.
− Symbolic Aggregate approXimation was proposed by Jessica Lin et al at the
University of California –Riverside;
− http://www.cs.ucr.edu/~eamonn/SAX.htm .
− It extends Piecewise Aggregate Approximation (PAA) symbolic representation
approach.
− SAX algorithm is interesting for in-network processing in WSN because of its
simplicity and low computational complexity.
− SAX provides reasonable sensitivity and selectivity in representing the data.
− The use of a symbolic representation makes it possible to use several other
algorithms and techniques to process/utilise SAX representations such as hashing,
pattern matching, suffix trees etc.
Processing steps in SAX
− SAX transforms a time-series X of length n into the string of
arbitrary length, where typically, using an alphabet A of size a >
2.
− The SAX algorithm has two main steps:
−Transforming the original time-series into a PAA
representation
−Converting the PAA intermediate representation into a
string during.
− The string representations can be used for pattern matching,
distance measurements, outlier detection, etc.
Piecewise Aggregate Approximation
− In PAA, to reduce the time series from n dimensions to w
dimensions, the data is divided into w equal sized “frames.”
− The mean value of the data falling within a frame is calculated
and a vector of these values becomes the data-reduced
representation.
− Before applying PAA, each time series to have a needs to be
normalised to achieve a mean of zero and a standard
deviation of one.
−The reason is to avoid comparing time series with different
offsets and amplitudes;
Source: Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003. A symbolic representation of time series, with implications for streaming algorithms. In
Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery (DMKD '03). ACM, New York, NY, USA, 2-11.
Normalisation in SAX
− The SAX algorithm starts with dividing the data into frames
and then normalizing the input data.
− It uses a z-normalisation method.
− SAX then applies PAA to normalised segments
− and then assigns alphabets to each frame according to the
breakpoints.
31
Normal Distribution
−The "standard normal distribution" is a
distribution given by:
−Mean (μ)=0 and
−Standard deviation (σ)=1
−The normal (or Gaussian) distribution is a
continuous probability distribution.
32
Normal or Gaussian Distribution
−“The central limit theorem states that under certain
(fairly common) conditions, the sum of many random
variables will have an approximately normal
distribution”.
−This means under fairly common condition and over
long-term observation most of the continuous and
time-series data will follow Gaussian distribution.
33
Source: Wikipedia
Standard Normal Distribution
34Image source: Mathworld.wolfram.com
Probability Density Functions
35
Further reading: https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/random-
variables-continuous/v/probability-density-functions
Z Normalisation
− Z Normalisation converts all values in a time series to a
common scale with an average of zero and standard deviation
of one (Gaussian distribution).
Normalised: zi = (ci – )/μ σ
: Standard deviationσ
: Meanμ
ci : item i in the list
36
SAX- normalisation before PAA
Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1
Mean (μ): =μ (2+3+4.5+7.6+4+2+2+2+3+1)/10= 3.11
Standard deviation (σ):
(2-3.11)2
= 1.2321
(3-3.11)2
= 0.0121
(4.5-3.11)2
= 1.9321
(7.6-3.11)2
= 20.1601
(4-3.11)2
= 0.7921
(2-3.11)2
= 1.2321
(2-3.11)2
= 1.2321
(2-3.11)2
= 1.2321
(3-3.11)2
= 0.0121
(1-3.11)2
= 4.4521
1.2321+0.0121+ 1.9321+ 20.1601+
0.7921+ 1.2321+ 1.2321+ 1.2321+
0.0121+4.4521 = 33.5211
= √ (33.5211/10) = 1.79691402131σ
Normalisation
Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1
Normalised: zi = (ci – )/μ σ
= 1.79691402131σ
= 3.11μ
z1 = (2- 3.11)/1.79691402131 = -0.617725716
z2 = (3-3.11)/ 1.79691402131= -0.061216062
z3 = (4.5-3.11)/ 1.79691402131= 0.773548419
z4 = (7.6-3.11)/ 1.79691402131= 2.498728346
z5 = (4-3.11)/ 1.79691402131= 0.495293592
z6 = (2-3.11)/ 1.79691402131= -0.617725716
z7 = (2-3.11)/ 1.79691402131= -0.617725716
z8 = (2-3.11)/ 1.79691402131= -0.617725716
z9 = (3-3.11)/ 1.79691402131= -0.061216062
z10 = (1-3.11)/ 1.79691402131= -1.17423537
NormalisedTime series (z): -0.617725716, -0.061216062, 0.773548419,
2.498728346, 0.495293592, -0.617725716, -0.617725716, -0.617725716,
-0.061216062, -1.17423537
PAA calculation
Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1
Normalised Timeseries (z): -0.617725716, -0.061216062,
0.773548419, 2.498728346, 0.495293592, -0.617725716,
-0.617725716, -0.617725716, -0.061216062, -1.17423537
PAA (w=5): -0.339470889, 1.636138382, -0.061216062,
-0.617725716, -0.617725716
PAA to SAX Conversion
− Conversion of the PAA representation of a time-series into
SAX is based on producing symbols that correspond to the
time-series features with equal probability.
− The SAX developers have shown that time-series which are
normalised (zero mean and standard deviation of 1) follow a
Normal distribution (Gaussian distribution).
− The SAX method introduces breakpoints that divides the
PAA representation to equal sections and assigns an alphabet
for each section.
− For defining breakpoints, Normal inverse cumulative distribution
function
Breakpoints in SAX
− “Breakpoints: breakpoints are a sorted list of numbers B = β 1,…, β a-1 such
that the area under a N(0,1) Gaussian curve from βi to βi+1 = 1/a”.
Source: Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003. A symbolic representation of time series, with implications for streaming algorithms. In Proceedings of the
8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery (DMKD '03). ACM, New York, NY, USA, 2-11.
Alphabet representation in SAX
− Let’s assume that we will have 4 symbols alphabet: a,b,c,d
− As shown in the table in the previous slide, the cut lines for this alphabet
(also shown as the thin red lines on the plot below) will be
{ -0.67, 0, 0.67 }
Source: JMOTIF Time series mining, http://code.google.com/p/jmotif/wiki/SAX
For example (3 eqi-probable areas)
43
β1 = -0.43 β2 = +0.43
SAX Represetantion
Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1
Normalised Timeseries (z): -0.617725716, -0.061216062,
0.773548419, 2.498728346, 0.495293592, -0.617725716,
-0.617725716, -0.617725716, -0.061216062, -1.17423537
PAA (w=5): -0.339470889, 1.636138382, -0.061216062,
-0.617725716, -0.617725716
Cut off ranges: {-0.67, 0, 0.67}
Alphabet: a ,b ,c, d
SAX representation: bdbbb
Features of the SAX technique
− SAX divides a time series data into equal segments and then
creates a string representation for each segment.
− The SAX patterns create the lower-level abstractions that are
used to create the higher-level interpretation of the
underlying data.
− The string representation of the SAX mechanism enables to
compare the patterns using a specific type of string similarity
function.
Distance Measures
− One of the common distance measures for time series is the
Euclidean distance.
− Given two time series Q and C of the same length n:
− If we transform the original sub-sequences into PAA
representations, and :
46
Distance function in SAX
− If we transform the data into the SAX representation, we can
define a MINDIST function that returns the minimum distance
between the original time series of two words:
47
Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
− The dist() function can be implemented using a lookup table;
− The value in cell (r,c) for any lookup table can be calculated by
the following expression.
48
Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
SAX Alphabet Distances
− Example for 4 alphabets
49Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
Distance function – Sample MATLAB code
50
% source: J. Lin et. al, https://cs.gmu.edu/~jessica/sax.htm
function dist_matrix = build_dist_table(alphabet_size)
switch alphabet_size
case 2, cutlines = [0];
case 3, cutlines = [-0.43 0.43];
case 4, cutlines = [-0.67 0 0.67];
case 5, cutlines = [-0.84 -0.25 0.25 0.84];
case 6, cutlines = [-0.97 -0.43 0 0.43 0.97];
case 7, cutlines = [-1.07 -0.57 -0.18 0.18 0.57 1.07];
case 8, cutlines = [-1.15 -0.67 -0.32 0 0.32 0.67 1.15];
case 9, cutlines = [-1.22 -0.76 -0.43 -0.14 0.14 0.43 0.76 1.22];
case 10, cutlines = [-1.28 -0.84 -0.52 -0.25 0. 0.25 0.52 0.84 1.28];
otherwise, disp('WARNING:: Alphabet size too big');
end;
dist_matrix=zeros(alphabet_size,alphabet_size);
for i = 1 : alphabet_size
% the min_dist for adjacent symbols are 0, so we start with i+2
for j = i+2 : alphabet_size
% square the distance now for future use
dist_matrix(i,j)=(cutlines(i)-cutlines(j-1))^2;
% the distance matrix is symmetric
dist_matrix(j,i) = dist_matrix(i,j);
end;
end;
Change Detection
− What?
− It refers to identifying times in which variations in the statistical
properties (e.g. mean, variance) of data streams are detected as soon
as they occur.
− Applications?
− Fire/fault detection
− Automatic segmentation
− Activity recognition
− Environmental monitoring
− Quality Control
− Challenges?
− On-line/adaptive change detection
51
Example
Source: Y.Fathy, P.Barnaghi, R. Tafazolli, “An On-line Adaptive Algorithm for Change Detection in Streaming Sensory Data”, IEEE System Journal, 2018.
Change Detection Techniques
− A key technique is Page’s Cumulative Sum (CUSUM) [1] [2]
− Cumulative differences between successive values.
− One-sided form: detects changes in one direction (e.g. increase or
decrease)
[1] Page, E. S. (1954). Continuous inspection schemes. Biometrika, 41(1/2), 100-115.
[2] Granjon, P. (2013).The CuSum algorithm-a small review.
Change Detection Techniques
− Two-sided form: detects changes in two direction (e.g. increase and
decrease)
Using change detection and model sharing
in sensor networks
− Data Reduction prediction-based approach lowers the number of
transmission from data source (i.e. sensor nodes) to a sink node.
− Prediction model runs simultaneously on sink and sensor nodes
− Only transmit if the error between the predicted and the actual
value (i.e. error) > emax
Highlights (with a cross) the readings which the node has to report to the
sink node to guarantee an accuracy +/-emax
Further reading: Santini, S., & Romer, K. (2006, June). An adaptive strategy for quality-based data reduction in wireless
sensor networks. In Proceedings of the 3rd international conference on networked sensing systems (INSS 2006) (pp. 29-36).
Prediction sharing model and energy efficiency in
sensor networks- multi sharing
Source: Y.Fathy, P.barnaghi, R.Tafazolli., “An Adaptive Method for Data Reduction in the Internet of Things”, IEEE World Forum on Internet of Things, 2018
57
Interpretation of data
− A primary goal of interconnecting devices and
collecting/processing data from them is to create situation
awareness and enable applications, machines, and human users
to better understand their surrounding environments.
− The understanding of a situation, or context, potentially
enables services and applications to make intelligent decisions
and to respond to the dynamics of their environments.
− Next week, more on annotation and interpretation of data,.
Quiz
− Consider this sensor measurements from a
stream:
− C = 2,3,5,0,1,3,2,0,0
− Calculate the normalised time series.
− Calculate PAA (w=3)
− Calculate SAX (alphabet size =3)
Acknowledgements
− Some parts of the content are adapted from:
− Holger Karl,Andreas Willig, Protocols and Architectures for Wireless Sensor
Networks, Protocols and Architectures for Wireless Sensor Networks,
chapters 3 and 12,Wiley, 2005 .
− Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003.A symbolic
representation of time series, with implications for streaming algorithms. In
Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining
and knowledge discovery (DMKD '03).ACM, NewYork, NY, USA, 2-11.
− JMOTIF Time series mining, http://code.google.com/p/jmotif/wiki/SAX
60
Questions?

More Related Content

What's hot

IoT Communication Protocols
IoT Communication ProtocolsIoT Communication Protocols
IoT Communication Protocols
Pradeep Kumar TS
 
Introduction to IoT Architectures and Protocols
Introduction to IoT Architectures and ProtocolsIntroduction to IoT Architectures and Protocols
Introduction to IoT Architectures and Protocols
Abdullah Alfadhly
 
What is SCADA system? SCADA Solutions for IoT
What is SCADA system? SCADA Solutions for IoTWhat is SCADA system? SCADA Solutions for IoT
What is SCADA system? SCADA Solutions for IoT
Embitel Technologies (I) PVT LTD
 
Iot
IotIot
Iot architecture
Iot architectureIot architecture
Iot architecture
Niranjan Kumar
 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolution
Sathvik N Prasad
 
Sdn ppt
Sdn pptSdn ppt
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
Selvaraj Seerangan
 
Unit 4
Unit 4Unit 4
Internet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M CommunicationInternet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M Communication
Sherin C Abraham
 
Internet of things (IOT) connects physical to digital
Internet of things (IOT) connects physical to digitalInternet of things (IOT) connects physical to digital
Internet of things (IOT) connects physical to digital
Eslam Nader
 
IOT - Unit 3.pptx
IOT - Unit 3.pptxIOT - Unit 3.pptx
IOT - Unit 3.pptx
PallaviPatil580306
 
15CS81- IoT Module-2
15CS81- IoT Module-215CS81- IoT Module-2
15CS81- IoT Module-2
Syed Mustafa
 
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | EdurekaIoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
Edureka!
 
Physical design of io t
Physical design of io tPhysical design of io t
Physical design of io t
ShilpaKrishna6
 
Iot logical design
Iot logical designIot logical design
Iot logical design
ShilpaKrishna6
 
IoT Connectivity
IoT Connectivity IoT Connectivity
IoT Connectivity
Hitesh Mohapatra
 
Iot and cloud computing
Iot and cloud computingIot and cloud computing
Iot and cloud computing
eteshagarwal1
 
4. Internet of Things - Reference Model and Architecture
4. Internet of Things - Reference Model and Architecture4. Internet of Things - Reference Model and Architecture
4. Internet of Things - Reference Model and Architecture
Jitendra Tomar
 
Internet of Things (IOT) - Technology and Applications
Internet of Things (IOT) - Technology and ApplicationsInternet of Things (IOT) - Technology and Applications
Internet of Things (IOT) - Technology and Applications
Dr. Mazlan Abbas
 

What's hot (20)

IoT Communication Protocols
IoT Communication ProtocolsIoT Communication Protocols
IoT Communication Protocols
 
Introduction to IoT Architectures and Protocols
Introduction to IoT Architectures and ProtocolsIntroduction to IoT Architectures and Protocols
Introduction to IoT Architectures and Protocols
 
What is SCADA system? SCADA Solutions for IoT
What is SCADA system? SCADA Solutions for IoTWhat is SCADA system? SCADA Solutions for IoT
What is SCADA system? SCADA Solutions for IoT
 
Iot
IotIot
Iot
 
Iot architecture
Iot architectureIot architecture
Iot architecture
 
The Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolutionThe Internet of Things (IoT) and its evolution
The Internet of Things (IoT) and its evolution
 
Sdn ppt
Sdn pptSdn ppt
Sdn ppt
 
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
Unit 2,3,4 _ Internet of Things A Hands-On Approach (Arshdeep Bahga, Vijay Ma...
 
Unit 4
Unit 4Unit 4
Unit 4
 
Internet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M CommunicationInternet of Things with Cloud Computing and M2M Communication
Internet of Things with Cloud Computing and M2M Communication
 
Internet of things (IOT) connects physical to digital
Internet of things (IOT) connects physical to digitalInternet of things (IOT) connects physical to digital
Internet of things (IOT) connects physical to digital
 
IOT - Unit 3.pptx
IOT - Unit 3.pptxIOT - Unit 3.pptx
IOT - Unit 3.pptx
 
15CS81- IoT Module-2
15CS81- IoT Module-215CS81- IoT Module-2
15CS81- IoT Module-2
 
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | EdurekaIoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
IoT Technology Tutorial | IoT Technology Stack | IoT Project Hands-On | Edureka
 
Physical design of io t
Physical design of io tPhysical design of io t
Physical design of io t
 
Iot logical design
Iot logical designIot logical design
Iot logical design
 
IoT Connectivity
IoT Connectivity IoT Connectivity
IoT Connectivity
 
Iot and cloud computing
Iot and cloud computingIot and cloud computing
Iot and cloud computing
 
4. Internet of Things - Reference Model and Architecture
4. Internet of Things - Reference Model and Architecture4. Internet of Things - Reference Model and Architecture
4. Internet of Things - Reference Model and Architecture
 
Internet of Things (IOT) - Technology and Applications
Internet of Things (IOT) - Technology and ApplicationsInternet of Things (IOT) - Technology and Applications
Internet of Things (IOT) - Technology and Applications
 

Similar to Lecture 6: IoT Data Processing

Internet of Things: Concepts and Technologies
Internet of Things: Concepts and TechnologiesInternet of Things: Concepts and Technologies
Internet of Things: Concepts and Technologies
PayamBarnaghi
 
Wireless sensor network applications
Wireless sensor network applicationsWireless sensor network applications
Wireless sensor network applications
Deepshika Reddy
 
IRJET- Energy Efficient Protocol in Wireless Sensor Network
IRJET- Energy Efficient Protocol in Wireless Sensor NetworkIRJET- Energy Efficient Protocol in Wireless Sensor Network
IRJET- Energy Efficient Protocol in Wireless Sensor Network
IRJET Journal
 
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
IRJET Journal
 
Ccna exploration network fundamentals
Ccna exploration  network fundamentalsCcna exploration  network fundamentals
Ccna exploration network fundamentals
IT Tech
 
Crowd management system
Crowd management systemCrowd management system
Crowd management system
Mumbaikar Le
 
Enhancing Data Transmission and Protection in Wireless Sensor Node- A Review
Enhancing Data Transmission and Protection in Wireless Sensor Node- A ReviewEnhancing Data Transmission and Protection in Wireless Sensor Node- A Review
Enhancing Data Transmission and Protection in Wireless Sensor Node- A Review
IRJET Journal
 
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
journalBEEI
 
Ijarcet vol-2-issue-2-756-760
Ijarcet vol-2-issue-2-756-760Ijarcet vol-2-issue-2-756-760
Ijarcet vol-2-issue-2-756-760
Editor IJARCET
 
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
paperpublications3
 
June 28 Presentation
June 28 PresentationJune 28 Presentation
June 28 Presentation
Andrew McGarry
 
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor NetworksA Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
IRJET Journal
 
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
IRJET Journal
 
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
IOSR Journals
 
wireless sensor network
wireless sensor networkwireless sensor network
wireless sensor network
DeepaDasarathan
 
A servey on wireless mesh networking module
A servey on wireless mesh networking moduleA servey on wireless mesh networking module
A servey on wireless mesh networking module
eSAT Journals
 
Design of a Wireless Sensor Network from an Energy Management Perspective
Design of a Wireless Sensor Network from an Energy Management PerspectiveDesign of a Wireless Sensor Network from an Energy Management Perspective
Design of a Wireless Sensor Network from an Energy Management Perspective
?? ?
 
Sensor net
Sensor netSensor net
Sensor net
Ines Bahram
 
2015 11-07 -ad_hoc__network architectures and protocol stack
2015 11-07 -ad_hoc__network architectures and protocol stack2015 11-07 -ad_hoc__network architectures and protocol stack
2015 11-07 -ad_hoc__network architectures and protocol stack
Syed Ariful Islam Emon
 
S04404116120
S04404116120S04404116120
S04404116120
IJERA Editor
 

Similar to Lecture 6: IoT Data Processing (20)

Internet of Things: Concepts and Technologies
Internet of Things: Concepts and TechnologiesInternet of Things: Concepts and Technologies
Internet of Things: Concepts and Technologies
 
Wireless sensor network applications
Wireless sensor network applicationsWireless sensor network applications
Wireless sensor network applications
 
IRJET- Energy Efficient Protocol in Wireless Sensor Network
IRJET- Energy Efficient Protocol in Wireless Sensor NetworkIRJET- Energy Efficient Protocol in Wireless Sensor Network
IRJET- Energy Efficient Protocol in Wireless Sensor Network
 
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
IRJET-Structure less Efficient Data Aggregation and Data Integrity in Sensor ...
 
Ccna exploration network fundamentals
Ccna exploration  network fundamentalsCcna exploration  network fundamentals
Ccna exploration network fundamentals
 
Crowd management system
Crowd management systemCrowd management system
Crowd management system
 
Enhancing Data Transmission and Protection in Wireless Sensor Node- A Review
Enhancing Data Transmission and Protection in Wireless Sensor Node- A ReviewEnhancing Data Transmission and Protection in Wireless Sensor Node- A Review
Enhancing Data Transmission and Protection in Wireless Sensor Node- A Review
 
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
Data Flow in Wireless Sensor Network Protocol Stack by using Bellman-Ford Rou...
 
Ijarcet vol-2-issue-2-756-760
Ijarcet vol-2-issue-2-756-760Ijarcet vol-2-issue-2-756-760
Ijarcet vol-2-issue-2-756-760
 
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
Design of a Reliable Wireless Sensor Network with Optimized Energy Efficiency...
 
June 28 Presentation
June 28 PresentationJune 28 Presentation
June 28 Presentation
 
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor NetworksA Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
A Study on Quality Of Service (QOS) in Ubiquitous Wireless Sensor Networks
 
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
Multiple Sink Positioning and Relocation for Improving Lifetime in Wireless S...
 
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
Energy saving in P2P oriented Wireless Sensor Network (WSN) using the approac...
 
wireless sensor network
wireless sensor networkwireless sensor network
wireless sensor network
 
A servey on wireless mesh networking module
A servey on wireless mesh networking moduleA servey on wireless mesh networking module
A servey on wireless mesh networking module
 
Design of a Wireless Sensor Network from an Energy Management Perspective
Design of a Wireless Sensor Network from an Energy Management PerspectiveDesign of a Wireless Sensor Network from an Energy Management Perspective
Design of a Wireless Sensor Network from an Energy Management Perspective
 
Sensor net
Sensor netSensor net
Sensor net
 
2015 11-07 -ad_hoc__network architectures and protocol stack
2015 11-07 -ad_hoc__network architectures and protocol stack2015 11-07 -ad_hoc__network architectures and protocol stack
2015 11-07 -ad_hoc__network architectures and protocol stack
 
S04404116120
S04404116120S04404116120
S04404116120
 

More from PayamBarnaghi

Academic Research: A Survival Guide
Academic Research: A Survival GuideAcademic Research: A Survival Guide
Academic Research: A Survival Guide
PayamBarnaghi
 
Reproducibility in machine learning
Reproducibility in machine learningReproducibility in machine learning
Reproducibility in machine learning
PayamBarnaghi
 
Search, Discovery and Analysis of Sensory Data Streams
Search, Discovery and Analysis of Sensory Data StreamsSearch, Discovery and Analysis of Sensory Data Streams
Search, Discovery and Analysis of Sensory Data Streams
PayamBarnaghi
 
Internet Search: the past, present and the future
Internet Search: the past, present and the futureInternet Search: the past, present and the future
Internet Search: the past, present and the future
PayamBarnaghi
 
Scientific and Academic Research: A Survival Guide 
Scientific and Academic Research:  A Survival Guide Scientific and Academic Research:  A Survival Guide 
Scientific and Academic Research: A Survival Guide 
PayamBarnaghi
 
Lecture 8: IoT System Models and Applications
Lecture 8: IoT System Models and ApplicationsLecture 8: IoT System Models and Applications
Lecture 8: IoT System Models and Applications
PayamBarnaghi
 
Lecture 7: Semantic Technologies and Interoperability
Lecture 7: Semantic Technologies and InteroperabilityLecture 7: Semantic Technologies and Interoperability
Lecture 7: Semantic Technologies and Interoperability
PayamBarnaghi
 
Lecture 5: Software platforms and services
Lecture 5: Software platforms and services Lecture 5: Software platforms and services
Lecture 5: Software platforms and services
PayamBarnaghi
 
Internet of Things for healthcare: data integration and security/privacy issu...
Internet of Things for healthcare: data integration and security/privacy issu...Internet of Things for healthcare: data integration and security/privacy issu...
Internet of Things for healthcare: data integration and security/privacy issu...
PayamBarnaghi
 
Scientific and Academic Research: A Survival Guide 
Scientific and Academic Research:  A Survival Guide Scientific and Academic Research:  A Survival Guide 
Scientific and Academic Research: A Survival Guide 
PayamBarnaghi
 
Semantic Technolgies for the Internet of Things
Semantic Technolgies for the Internet of ThingsSemantic Technolgies for the Internet of Things
Semantic Technolgies for the Internet of Things
PayamBarnaghi
 
Internet of Things and Data Analytics for Smart Cities and eHealth
Internet of Things and Data Analytics for Smart Cities and eHealthInternet of Things and Data Analytics for Smart Cities and eHealth
Internet of Things and Data Analytics for Smart Cities and eHealth
PayamBarnaghi
 
Spatial Data on the Web
Spatial Data on the WebSpatial Data on the Web
Spatial Data on the Web
PayamBarnaghi
 
IoT-Lite: A Lightweight Semantic Model for the Internet of Things
IoT-Lite:  A Lightweight Semantic Model for the Internet of ThingsIoT-Lite:  A Lightweight Semantic Model for the Internet of Things
IoT-Lite: A Lightweight Semantic Model for the Internet of Things
PayamBarnaghi
 
The Future is Cyber-Healthcare
The Future is Cyber-Healthcare The Future is Cyber-Healthcare
The Future is Cyber-Healthcare
PayamBarnaghi
 
How to make cities "smarter"?
How to make cities "smarter"?How to make cities "smarter"?
How to make cities "smarter"?
PayamBarnaghi
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
PayamBarnaghi
 
Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things
PayamBarnaghi
 
Smart Cities….Smart Future
Smart Cities….Smart FutureSmart Cities….Smart Future
Smart Cities….Smart Future
PayamBarnaghi
 
What makes smart cities “Smart”?
What makes smart cities “Smart”? What makes smart cities “Smart”?
What makes smart cities “Smart”?
PayamBarnaghi
 

More from PayamBarnaghi (20)

Academic Research: A Survival Guide
Academic Research: A Survival GuideAcademic Research: A Survival Guide
Academic Research: A Survival Guide
 
Reproducibility in machine learning
Reproducibility in machine learningReproducibility in machine learning
Reproducibility in machine learning
 
Search, Discovery and Analysis of Sensory Data Streams
Search, Discovery and Analysis of Sensory Data StreamsSearch, Discovery and Analysis of Sensory Data Streams
Search, Discovery and Analysis of Sensory Data Streams
 
Internet Search: the past, present and the future
Internet Search: the past, present and the futureInternet Search: the past, present and the future
Internet Search: the past, present and the future
 
Scientific and Academic Research: A Survival Guide 
Scientific and Academic Research:  A Survival Guide Scientific and Academic Research:  A Survival Guide 
Scientific and Academic Research: A Survival Guide 
 
Lecture 8: IoT System Models and Applications
Lecture 8: IoT System Models and ApplicationsLecture 8: IoT System Models and Applications
Lecture 8: IoT System Models and Applications
 
Lecture 7: Semantic Technologies and Interoperability
Lecture 7: Semantic Technologies and InteroperabilityLecture 7: Semantic Technologies and Interoperability
Lecture 7: Semantic Technologies and Interoperability
 
Lecture 5: Software platforms and services
Lecture 5: Software platforms and services Lecture 5: Software platforms and services
Lecture 5: Software platforms and services
 
Internet of Things for healthcare: data integration and security/privacy issu...
Internet of Things for healthcare: data integration and security/privacy issu...Internet of Things for healthcare: data integration and security/privacy issu...
Internet of Things for healthcare: data integration and security/privacy issu...
 
Scientific and Academic Research: A Survival Guide 
Scientific and Academic Research:  A Survival Guide Scientific and Academic Research:  A Survival Guide 
Scientific and Academic Research: A Survival Guide 
 
Semantic Technolgies for the Internet of Things
Semantic Technolgies for the Internet of ThingsSemantic Technolgies for the Internet of Things
Semantic Technolgies for the Internet of Things
 
Internet of Things and Data Analytics for Smart Cities and eHealth
Internet of Things and Data Analytics for Smart Cities and eHealthInternet of Things and Data Analytics for Smart Cities and eHealth
Internet of Things and Data Analytics for Smart Cities and eHealth
 
Spatial Data on the Web
Spatial Data on the WebSpatial Data on the Web
Spatial Data on the Web
 
IoT-Lite: A Lightweight Semantic Model for the Internet of Things
IoT-Lite:  A Lightweight Semantic Model for the Internet of ThingsIoT-Lite:  A Lightweight Semantic Model for the Internet of Things
IoT-Lite: A Lightweight Semantic Model for the Internet of Things
 
The Future is Cyber-Healthcare
The Future is Cyber-Healthcare The Future is Cyber-Healthcare
The Future is Cyber-Healthcare
 
How to make cities "smarter"?
How to make cities "smarter"?How to make cities "smarter"?
How to make cities "smarter"?
 
The Internet of Things: What's next?
The Internet of Things: What's next? The Internet of Things: What's next?
The Internet of Things: What's next?
 
Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things Information Engineering in the Age of the Internet of Things
Information Engineering in the Age of the Internet of Things
 
Smart Cities….Smart Future
Smart Cities….Smart FutureSmart Cities….Smart Future
Smart Cities….Smart Future
 
What makes smart cities “Smart”?
What makes smart cities “Smart”? What makes smart cities “Smart”?
What makes smart cities “Smart”?
 

Recently uploaded

What is Packaging of Products in Odoo 17
What is Packaging of Products in Odoo 17What is Packaging of Products in Odoo 17
What is Packaging of Products in Odoo 17
Celine George
 
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
thanhluan21
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
DrRavindrakshirsagar1
 
The Cruelty of Animal Testing in the Industry.pdf
The Cruelty of Animal Testing in the Industry.pdfThe Cruelty of Animal Testing in the Industry.pdf
The Cruelty of Animal Testing in the Industry.pdf
luzmilaglez334
 
How to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POSHow to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POS
Celine George
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
artenzmartenkai
 
How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17
Celine George
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
Scholarhat
 
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.docBài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
PhngThLmHnh
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
thanhluan21
 
New Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 SlidesNew Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 Slides
Celine George
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
DrRavindrakshirsagar1
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
lakitawilson
 
How to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 WebsiteHow to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 Website
Celine George
 
Allopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation PowerpointAllopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation Powerpoint
Julie Sarpy
 
How To Create a Transient Model in Odoo 17
How To Create a Transient Model in Odoo 17How To Create a Transient Model in Odoo 17
How To Create a Transient Model in Odoo 17
Celine George
 
RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
Murugan Solaiyappan
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
Celine George
 
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
AnneMarieJacildo
 
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
Celine George
 

Recently uploaded (20)

What is Packaging of Products in Odoo 17
What is Packaging of Products in Odoo 17What is Packaging of Products in Odoo 17
What is Packaging of Products in Odoo 17
 
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY N...
 
Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.Genetics Teaching Plan: Dr.Kshirsagar R.V.
Genetics Teaching Plan: Dr.Kshirsagar R.V.
 
The Cruelty of Animal Testing in the Industry.pdf
The Cruelty of Animal Testing in the Industry.pdfThe Cruelty of Animal Testing in the Industry.pdf
The Cruelty of Animal Testing in the Industry.pdf
 
How to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POSHow to Manage Large Scrollbar in Odoo 17 POS
How to Manage Large Scrollbar in Odoo 17 POS
 
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptxKesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
Kesadaran_Berbangsa_dan_Bernegara_Nasion.pptx
 
How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17How To Update One2many Field From OnChange of Field in Odoo 17
How To Update One2many Field From OnChange of Field in Odoo 17
 
C# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdfC# Interview Questions PDF By ScholarHat.pdf
C# Interview Questions PDF By ScholarHat.pdf
 
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.docBài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
Bài tập bộ trợ anh 7 I learn smart world kì 1 năm học 2022 2023 unit 1.doc
 
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ..."DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
"DANH SÁCH THÍ SINH XÉT TUYỂN SỚM ĐỦ ĐIỀU KIỆN TRÚNG TUYỂN ĐẠI HỌC CHÍNH QUY ...
 
New Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 SlidesNew Features in Odoo 17 Sign - Odoo 17 Slides
New Features in Odoo 17 Sign - Odoo 17 Slides
 
formative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.Vformative Evaluation By Dr.Kshirsagar R.V
formative Evaluation By Dr.Kshirsagar R.V
 
NAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource BookNAEYC Code of Ethical Conduct Resource Book
NAEYC Code of Ethical Conduct Resource Book
 
How to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 WebsiteHow to Create & Publish a Blog in Odoo 17 Website
How to Create & Publish a Blog in Odoo 17 Website
 
Allopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation PowerpointAllopathic M1 Srudent Orientation Powerpoint
Allopathic M1 Srudent Orientation Powerpoint
 
How To Create a Transient Model in Odoo 17
How To Create a Transient Model in Odoo 17How To Create a Transient Model in Odoo 17
How To Create a Transient Model in Odoo 17
 
RDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEWRDBMS Lecture Notes Unit4 chapter12 VIEW
RDBMS Lecture Notes Unit4 chapter12 VIEW
 
How to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POSHow to Manage Line Discount in Odoo 17 POS
How to Manage Line Discount in Odoo 17 POS
 
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
1-NLC-MATH7-Consolidation-Lesson1 2024.pptx
 
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17How to Manage Shipping Connectors & Shipping Methods in Odoo 17
How to Manage Shipping Connectors & Shipping Methods in Odoo 17
 

Lecture 6: IoT Data Processing

  • 1. 1 Lecture 6: IoT Data Processing EEEM048/COM3023- Internet of Things Prof. Payam Barnaghi Centre for Vision, Speech and Signal Processing (CVSSP) Electrical and Electronic Engineering Department University of Surrey Autumn 2018
  • 2. Wireless Sensor (and Actuator) Networks Sink node Gateway Core network e.g. InternetGateway End-user Computer services - The networks typically run Low Power Devices - Consist of one or more sensors, could be different type of sensors (or actuators) Operating Systems? Services? Protocols? Protocols? In-node Data Processing Data Aggregation/ Fusion Inference/ Processing of IoT data
  • 3. 3 Key characteristics of IoT devices −Often inexpensive sensors (actuators) equipped with a radio transceiver for various applications, typically low data rate ~ 10-250 kbps (but not always- e.g. CCTV and video/sound data will require higher bandwidth/data rates). −Deployed in large numbers −The sensors may need to coordinate to perform a desired task. −The acquired information (periodic or event-based) is reported back to the information processing centre (in some cases in-network processing is required) or the information/alert is communicated to another device (M2M communication/interaction) −Solutions are often application-dependent. 3
  • 4. 4 Beyond conventional sensors −Human as a sensor (citizen sensors) −e.g. tweeting real world data and/or events −Software sensors −e.g. Software agents/services generating/representing data Road block, A3 Road block, A3
  • 5. 5 The benefits of data processing in IoT − Convert (billions of) smart meter readings to better predict and balance power consumption. − Analyze thousands of traffic, pollution, weather, congestion, public transport and event sensory data to provide better traffic and smart city management. − Monitor patients, in-home care and several other healthcare applications… − The applications require: real-time, reliable, efficient (for low power and resource limited nodes), and scalable solutions. Adapted from: What is Bog Data?, IBM
  • 6. 6 IoT Data Access − Publish/Subscribe (long-term/short-term) − Ad-hoc query − The typical types of data request for sensory data: − Query based on − ID (resource/service) – for known resources; eg. data from device ABC001? − Location e.g. traffic data from junction ABC01 − Type e.g. all the temperature readings − Time – requests for fresh or historical data; e.g. all the collected data at 9:00am − One of the above + a range [+ unit of measurement] − Type/Location/Time + A combination of Quality of Information attributes − An entity of interest (a feature of an entity on interest)- e.g. temperature of a room; quality of air in an area; − Complex DataTypes (e.g. pollution data could be a combination of different types)
  • 7. Sensor Data − The sensory data represents physical world observations and measurements and requires time and location and other descriptive attributes to make the data more meaningful. − For example, a temperature value of 15 degree will be more meaningful when it is described with spatial (e.g. Guildford town centre) and temporal (e.g. 8:15AM GMT, 07-11-2018), and unit (e.g. Celsius) attributes. − The sensory data can also include other detailed meta-data that describe quality or device related attributes (e.g. Precision, Accuracy). 7
  • 8. Sensor Data 15, C, 08:15, 51.243057, -0.589444 8
  • 9. Data Processing and Interpretation −Processing and Interpretation of data (this week) −Meta-data enhancement, annotation and semantically described IoT data (next week) 9
  • 10. IoT Data Processing WSN WSN WSN WSN WSN Network-enabled Devices Network-enabled Devices Network services/storage and processing units Data/service access at application level Data collections and processing within the networks Data Discovery Service/ Resource Discovery
  • 11. 11 In-network processing − Mobile Ad-hoc Networks can be seen as a set of nodes that deliver bits from one end to the other; − WSNs, on the other end, are expected to provide information, not necessarily original bits − Gives additional options − e.g., manipulate or process the data in the network − Main example: aggregation − Applying aggregation functions to a obtain an average value of measurement data − Typical functions: minimum, maximum, average, sum, … − Not amenable functions: median Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 12. 12 In-network processing − Depending on application, more sophisticated processing of data can take place within the network − Example edge detection: locally exchange raw data with neighboring nodes, compute edges, only communicate edge description to far away data sinks. − Example tracking/angle detection of signal source: conceive of sensor nodes as a distributed microphone array, use it to compute the angle of a single source, only communicate this angle, not all the raw data. − Use temporal and spatial correlations − Observed signals might vary only slowly in time; so no need to transmit all data at full rate all the time. − Signals of neighboring nodes are often quite similar; only try to transmit differences. Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 13. 13 Data-centric networking − In typical networks (including ad-hoc networks), network transactions are addressed to the identities of specific nodes − A “node-centric” or “address-centric” networking paradigm − In a redundantly deployed sensor network, specific source of an event, alarm, etc. might not be important − Redundancy: e.g., several nodes can observe the same area − Thus: focus networking transactions on the data directly instead of their senders and transmitters; i.e. data-centric networking − Specially this idea is reinforced by the fact that we might have multiple sources to provide information and observations from the same or similar areas. Source: Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks Holger Karl, Andreas Willig, chapter 3, Wiley, 2005 .
  • 14. Data Aggregation − Computing a smaller representation of a number of data items (or messages) that is extracted from all the individual data items. − For example computing min/max or mean of sensor data. − More advance aggregation solutions could use approximation techniques to transform high-dimensionality data to lower-dimensionality abstractions/representations. − The aggregated data can be smaller in size, represent patterns/abstractions; so in multi-hop networks, nodes can receive data form other node and aggregate them before forwarding them to a sink or gateway. − The aggregation can happen on a sink/gateway node.
  • 15. Aggregation example − Reduce number of transmitted bits/packets by applying an aggregation function in the network 1 1 3 1 1 6 1 1 1 1 1 1 Source: Holger Karl, Andreas Willig, Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks, chapter 3, Wiley, 2005 .
  • 16. Efficacy of an aggregation mechanism − Accuracy: difference between the resulting value or representation and the original data − Some solutions can be lossless or lossy depending on the applied techniques. − Completeness: the percentage of all the data items that are included in the computation of the aggregated data. − Latency: delay time to compute and report the aggregated data − Computation foot-print; complexity; − Overhead: the main advantage of the aggregation is reducing the size of the data representation; − Aggregation functions can trade-off between accuracy, latency and overhead; − Aggregation usually happens closer to the source.
  • 17. Publish/Subscribe − Achieved by publish/subscribe paradigm − Idea: Entities can publish data under certain names − Entities can subscribe to updates of such named data − Conceptually: Implemented by a software bus − Software bus stores subscriptions, published data; names used as filters; subscribers notified when values of named data changes Software bus Publisher 1 Publisher 2 Subscriber 1 Subscriber 2 Subscriber 3 Source: Holger Karl, Andreas Willig, Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks, chapter 12, Wiley, 2005 .
  • 18. Message Brokers − For example, RabbitMQ is one of the widely deployed open source message brokers. − Protocols such as AMQP, MQTT. − A producer is a user application that sends messages. − A queue is a buffer that stores messages. − A consumer is a user application that receives messages. 18 Source: https://www.rabbitmq.com/tutorials/tutorial-three-python.html
  • 19. MQTT Pub/Sub Protocol − MQTelemetry Transport (MQTT) is a lightweight broker-based publish/subscribe messaging protocol. − MQTT is designed to be open, simple, lightweight and easy to implement. − These characteristics make MQTT ideal for use in constrained environments, for example in IoT. −Where the network is expensive, has low bandwidth or is unreliable −When run on an embedded device with limited processor or memory resources; − A small transport overhead (the fixed-length header is just 2 bytes), and protocol exchanges minimised to reduce network traffic Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html
  • 20. MQTT Messages − The MQTT packet or message format consists of a 2 byte fixed header (always present) + Variable-header (not always present)+ payload (not always present)*. − It supports publish/subscribe message pattern to provide one- to-many message distribution and decoupling of applications − A messaging transport that is agnostic to the content of the payload − The use of TCP/IP to provide basic network connectivity 20* Source: http://www.steves-internet-guide.com/mqtt-protocol-messages-overview/
  • 21. MQTT- Quality of Service − Three qualities of service for message delivery: − "At most once", where messages are delivered according to the best efforts of the underlying TCP/IP network. Message loss or duplication can occur. −This level could be used, for example, with ambient sensor data where it does not matter if an individual reading is lost as the next one will be published soon after. − "At least once", where messages are assured to arrive but duplicates may occur. − "Exactly once", where message are assured to arrive exactly once.This level could be used, for example, with billing systems where duplicate or lost messages could lead to incorrect charges being applied. Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html
  • 22. MQTT Message Format − The message header for each MQTT command message contains a fixed header. − Some messages also require a variable header and a payload. − The format for each part of the message header: Source: MQTT V3.1 Protocol Specification, IBM, http://public.dhe.ibm.com/software/dw/webservices/ws-mqtt/mqtt-v3r1.html — DUP: Duplicate delivery — QoS: Quality of Service — RETAIN: RETAIN flag —This flag is only used on PUBLISH messages. When a client sends a PUBLISH to a server, if the Retain flag is set (1), the server should hold on to the message after it has been delivered to the current subscribers. —This allows new subscribers to instantly receive data with the retained or Last Known Good value.
  • 23. Sensor Data as time-series data − The sensor data (or IoT data in general) can be seen as time- series data. − A sensor stream refers to a source that provide sensor data over time. − The data can be sampled/collected at a rate (can be also variable) and is sent as a series of values. − Over time, there will be a large number of data items collected. − Using time-series processing techniques can help to reduce the size of the data that is communicated; −Let’s remember, communication can consume more energy than computaiton;
  • 24. Time series data and segmentation − The collection of several points spaced in time, having a temporal order is known as time series data.Time series can be analysed using various techniques such as clustering, classification, regression and prediction in the fields of data mining, machine learning, signal processing, communication engineering, and statistics. − Given a time series T containing n data points, segmentation is defined as the construction of a model T’, from l piecewise segments (l < n) such that T’ closely approximates T. 24
  • 25. Time series data 25 Segment 1 Segment 2 Segment …
  • 26. Sensor Data as time-series data − Different representation method that introduced for time-series data can be applied. − The goal is to reduce the dimensionality (and size) of the data, to find patterns, detect anomalies, to query similar data; − Dimensionality reduction techniques transform a data series with n items to a representation with w items where w < n. − This functions are often lossy in comparison with solutions like normal compression that preserve all the data. − One of these techniques is called Symbolic Aggregation Approximation (SAX). − SAX was originally proposed for symbolic representation of time-series data; it can be also used for symbolic representation of time-series sensor measurements. − The computational foot-print of SAX is low; so it can be also used as a an in-network processing technique.
  • 27. 27 In-network processing Using Symbolic Aggregate Approximation (SAX) SAX Pattern (blue) with word length of 20 and a vocabulary of 10 symbols over the original sensor time-series data (green) Source: P. Barnaghi, F. Ganz, C. Henson, A. Sheth, "Computing Perception from Sensor Data", in Proc. of the IEEE Sensors 2012, Oct. 2012. fggff jfhig fggff
  • 28. Symbolic Aggregate Approximation (SAX) − SAX transforms time-series data into symbolic string representations. − Symbolic Aggregate approXimation was proposed by Jessica Lin et al at the University of California –Riverside; − http://www.cs.ucr.edu/~eamonn/SAX.htm . − It extends Piecewise Aggregate Approximation (PAA) symbolic representation approach. − SAX algorithm is interesting for in-network processing in WSN because of its simplicity and low computational complexity. − SAX provides reasonable sensitivity and selectivity in representing the data. − The use of a symbolic representation makes it possible to use several other algorithms and techniques to process/utilise SAX representations such as hashing, pattern matching, suffix trees etc.
  • 29. Processing steps in SAX − SAX transforms a time-series X of length n into the string of arbitrary length, where typically, using an alphabet A of size a > 2. − The SAX algorithm has two main steps: −Transforming the original time-series into a PAA representation −Converting the PAA intermediate representation into a string during. − The string representations can be used for pattern matching, distance measurements, outlier detection, etc.
  • 30. Piecewise Aggregate Approximation − In PAA, to reduce the time series from n dimensions to w dimensions, the data is divided into w equal sized “frames.” − The mean value of the data falling within a frame is calculated and a vector of these values becomes the data-reduced representation. − Before applying PAA, each time series to have a needs to be normalised to achieve a mean of zero and a standard deviation of one. −The reason is to avoid comparing time series with different offsets and amplitudes; Source: Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003. A symbolic representation of time series, with implications for streaming algorithms. In Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery (DMKD '03). ACM, New York, NY, USA, 2-11.
  • 31. Normalisation in SAX − The SAX algorithm starts with dividing the data into frames and then normalizing the input data. − It uses a z-normalisation method. − SAX then applies PAA to normalised segments − and then assigns alphabets to each frame according to the breakpoints. 31
  • 32. Normal Distribution −The "standard normal distribution" is a distribution given by: −Mean (μ)=0 and −Standard deviation (σ)=1 −The normal (or Gaussian) distribution is a continuous probability distribution. 32
  • 33. Normal or Gaussian Distribution −“The central limit theorem states that under certain (fairly common) conditions, the sum of many random variables will have an approximately normal distribution”. −This means under fairly common condition and over long-term observation most of the continuous and time-series data will follow Gaussian distribution. 33 Source: Wikipedia
  • 34. Standard Normal Distribution 34Image source: Mathworld.wolfram.com
  • 35. Probability Density Functions 35 Further reading: https://www.khanacademy.org/math/statistics-probability/random-variables-stats-library/random- variables-continuous/v/probability-density-functions
  • 36. Z Normalisation − Z Normalisation converts all values in a time series to a common scale with an average of zero and standard deviation of one (Gaussian distribution). Normalised: zi = (ci – )/μ σ : Standard deviationσ : Meanμ ci : item i in the list 36
  • 37. SAX- normalisation before PAA Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1 Mean (μ): =μ (2+3+4.5+7.6+4+2+2+2+3+1)/10= 3.11 Standard deviation (σ): (2-3.11)2 = 1.2321 (3-3.11)2 = 0.0121 (4.5-3.11)2 = 1.9321 (7.6-3.11)2 = 20.1601 (4-3.11)2 = 0.7921 (2-3.11)2 = 1.2321 (2-3.11)2 = 1.2321 (2-3.11)2 = 1.2321 (3-3.11)2 = 0.0121 (1-3.11)2 = 4.4521 1.2321+0.0121+ 1.9321+ 20.1601+ 0.7921+ 1.2321+ 1.2321+ 1.2321+ 0.0121+4.4521 = 33.5211 = √ (33.5211/10) = 1.79691402131σ
  • 38. Normalisation Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1 Normalised: zi = (ci – )/μ σ = 1.79691402131σ = 3.11μ z1 = (2- 3.11)/1.79691402131 = -0.617725716 z2 = (3-3.11)/ 1.79691402131= -0.061216062 z3 = (4.5-3.11)/ 1.79691402131= 0.773548419 z4 = (7.6-3.11)/ 1.79691402131= 2.498728346 z5 = (4-3.11)/ 1.79691402131= 0.495293592 z6 = (2-3.11)/ 1.79691402131= -0.617725716 z7 = (2-3.11)/ 1.79691402131= -0.617725716 z8 = (2-3.11)/ 1.79691402131= -0.617725716 z9 = (3-3.11)/ 1.79691402131= -0.061216062 z10 = (1-3.11)/ 1.79691402131= -1.17423537 NormalisedTime series (z): -0.617725716, -0.061216062, 0.773548419, 2.498728346, 0.495293592, -0.617725716, -0.617725716, -0.617725716, -0.061216062, -1.17423537
  • 39. PAA calculation Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1 Normalised Timeseries (z): -0.617725716, -0.061216062, 0.773548419, 2.498728346, 0.495293592, -0.617725716, -0.617725716, -0.617725716, -0.061216062, -1.17423537 PAA (w=5): -0.339470889, 1.636138382, -0.061216062, -0.617725716, -0.617725716
  • 40. PAA to SAX Conversion − Conversion of the PAA representation of a time-series into SAX is based on producing symbols that correspond to the time-series features with equal probability. − The SAX developers have shown that time-series which are normalised (zero mean and standard deviation of 1) follow a Normal distribution (Gaussian distribution). − The SAX method introduces breakpoints that divides the PAA representation to equal sections and assigns an alphabet for each section. − For defining breakpoints, Normal inverse cumulative distribution function
  • 41. Breakpoints in SAX − “Breakpoints: breakpoints are a sorted list of numbers B = β 1,…, β a-1 such that the area under a N(0,1) Gaussian curve from βi to βi+1 = 1/a”. Source: Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003. A symbolic representation of time series, with implications for streaming algorithms. In Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery (DMKD '03). ACM, New York, NY, USA, 2-11.
  • 42. Alphabet representation in SAX − Let’s assume that we will have 4 symbols alphabet: a,b,c,d − As shown in the table in the previous slide, the cut lines for this alphabet (also shown as the thin red lines on the plot below) will be { -0.67, 0, 0.67 } Source: JMOTIF Time series mining, http://code.google.com/p/jmotif/wiki/SAX
  • 43. For example (3 eqi-probable areas) 43 β1 = -0.43 β2 = +0.43
  • 44. SAX Represetantion Timeseries (c): 2, 3, 4.5, 7.6, 4, 2, 2, 2, 3, 1 Normalised Timeseries (z): -0.617725716, -0.061216062, 0.773548419, 2.498728346, 0.495293592, -0.617725716, -0.617725716, -0.617725716, -0.061216062, -1.17423537 PAA (w=5): -0.339470889, 1.636138382, -0.061216062, -0.617725716, -0.617725716 Cut off ranges: {-0.67, 0, 0.67} Alphabet: a ,b ,c, d SAX representation: bdbbb
  • 45. Features of the SAX technique − SAX divides a time series data into equal segments and then creates a string representation for each segment. − The SAX patterns create the lower-level abstractions that are used to create the higher-level interpretation of the underlying data. − The string representation of the SAX mechanism enables to compare the patterns using a specific type of string similarity function.
  • 46. Distance Measures − One of the common distance measures for time series is the Euclidean distance. − Given two time series Q and C of the same length n: − If we transform the original sub-sequences into PAA representations, and : 46
  • 47. Distance function in SAX − If we transform the data into the SAX representation, we can define a MINDIST function that returns the minimum distance between the original time series of two words: 47 Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
  • 48. − The dist() function can be implemented using a lookup table; − The value in cell (r,c) for any lookup table can be calculated by the following expression. 48 Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
  • 49. SAX Alphabet Distances − Example for 4 alphabets 49Source: Lin et. al, http://www.cs.ucr.edu/~eamonn/SAX.pdf
  • 50. Distance function – Sample MATLAB code 50 % source: J. Lin et. al, https://cs.gmu.edu/~jessica/sax.htm function dist_matrix = build_dist_table(alphabet_size) switch alphabet_size case 2, cutlines = [0]; case 3, cutlines = [-0.43 0.43]; case 4, cutlines = [-0.67 0 0.67]; case 5, cutlines = [-0.84 -0.25 0.25 0.84]; case 6, cutlines = [-0.97 -0.43 0 0.43 0.97]; case 7, cutlines = [-1.07 -0.57 -0.18 0.18 0.57 1.07]; case 8, cutlines = [-1.15 -0.67 -0.32 0 0.32 0.67 1.15]; case 9, cutlines = [-1.22 -0.76 -0.43 -0.14 0.14 0.43 0.76 1.22]; case 10, cutlines = [-1.28 -0.84 -0.52 -0.25 0. 0.25 0.52 0.84 1.28]; otherwise, disp('WARNING:: Alphabet size too big'); end; dist_matrix=zeros(alphabet_size,alphabet_size); for i = 1 : alphabet_size % the min_dist for adjacent symbols are 0, so we start with i+2 for j = i+2 : alphabet_size % square the distance now for future use dist_matrix(i,j)=(cutlines(i)-cutlines(j-1))^2; % the distance matrix is symmetric dist_matrix(j,i) = dist_matrix(i,j); end; end;
  • 51. Change Detection − What? − It refers to identifying times in which variations in the statistical properties (e.g. mean, variance) of data streams are detected as soon as they occur. − Applications? − Fire/fault detection − Automatic segmentation − Activity recognition − Environmental monitoring − Quality Control − Challenges? − On-line/adaptive change detection 51
  • 52. Example Source: Y.Fathy, P.Barnaghi, R. Tafazolli, “An On-line Adaptive Algorithm for Change Detection in Streaming Sensory Data”, IEEE System Journal, 2018.
  • 53. Change Detection Techniques − A key technique is Page’s Cumulative Sum (CUSUM) [1] [2] − Cumulative differences between successive values. − One-sided form: detects changes in one direction (e.g. increase or decrease) [1] Page, E. S. (1954). Continuous inspection schemes. Biometrika, 41(1/2), 100-115. [2] Granjon, P. (2013).The CuSum algorithm-a small review.
  • 54. Change Detection Techniques − Two-sided form: detects changes in two direction (e.g. increase and decrease)
  • 55. Using change detection and model sharing in sensor networks − Data Reduction prediction-based approach lowers the number of transmission from data source (i.e. sensor nodes) to a sink node. − Prediction model runs simultaneously on sink and sensor nodes − Only transmit if the error between the predicted and the actual value (i.e. error) > emax Highlights (with a cross) the readings which the node has to report to the sink node to guarantee an accuracy +/-emax Further reading: Santini, S., & Romer, K. (2006, June). An adaptive strategy for quality-based data reduction in wireless sensor networks. In Proceedings of the 3rd international conference on networked sensing systems (INSS 2006) (pp. 29-36).
  • 56. Prediction sharing model and energy efficiency in sensor networks- multi sharing Source: Y.Fathy, P.barnaghi, R.Tafazolli., “An Adaptive Method for Data Reduction in the Internet of Things”, IEEE World Forum on Internet of Things, 2018
  • 57. 57 Interpretation of data − A primary goal of interconnecting devices and collecting/processing data from them is to create situation awareness and enable applications, machines, and human users to better understand their surrounding environments. − The understanding of a situation, or context, potentially enables services and applications to make intelligent decisions and to respond to the dynamics of their environments. − Next week, more on annotation and interpretation of data,.
  • 58. Quiz − Consider this sensor measurements from a stream: − C = 2,3,5,0,1,3,2,0,0 − Calculate the normalised time series. − Calculate PAA (w=3) − Calculate SAX (alphabet size =3)
  • 59. Acknowledgements − Some parts of the content are adapted from: − Holger Karl,Andreas Willig, Protocols and Architectures for Wireless Sensor Networks, Protocols and Architectures for Wireless Sensor Networks, chapters 3 and 12,Wiley, 2005 . − Jessica Lin, Eamonn Keogh, Stefano Lonardi, and Bill Chiu. 2003.A symbolic representation of time series, with implications for streaming algorithms. In Proceedings of the 8th ACM SIGMOD workshop on Research issues in data mining and knowledge discovery (DMKD '03).ACM, NewYork, NY, USA, 2-11. − JMOTIF Time series mining, http://code.google.com/p/jmotif/wiki/SAX