5th Annual International Cyber-Physical Cloud Computing Workshop, Washington DC, USA.

1. Physical-Cyber-Social Data Analytics &
Smart City Applications
1
Payam Barnaghi
Institute for Communication Systems (ICS)
University of Surrey
Guildford, United Kingdom
5th
Annual International Cyber-Physical Cloud
Computing Workshop

2. Cyber-Physical-Social Data
2P. Barnaghi et al., "Digital Technology Adoption in the Smart Built Environment", IET Sector Technical Briefing, The Institution of Engineering and Technology
(IET), I. Borthwick (editor), March 2015.

3. Internet of Things: The story so far
RFID based
solutions
Wireless Sensor and
Actuator networks
, solutions for
communication
technologies, energy
efficiency, routing, …
Smart Devices/
Web-enabled
Apps/Services, initial
products,
vertical applications, early
concepts and demos, …
Motion sensor
Motion sensor
ECG sensor
Physical-Cyber-Social
Systems, Linked-data,
semantics,
More products, more
heterogeneity,
solutions for control and
monitoring, …
Future: Cloud, Big (IoT) Data
Analytics, Interoperability,
Enhanced Cellular/Wireless Com.
for IoT, Real-world operational
use-cases and Industry and B2B
services/applications,
more Standards…
P. Barnaghi, A. Sheth, "Internet of Things: the story so far", IEEE IoT Newsletter, September 2014.
3

4. 4
“Each single data item is important.”
“Relying merely on data from sources that are
unevenly distributed, without considering
background information or social context, can
lead to imbalanced interpretations and
decisions.”
?

5. Data- Challenges
− Multi-modal and heterogeneous
− Noisy and incomplete
− Time and location dependent
− Dynamic and varies in quality
− Crowed sourced data can be unreliable
− Requires (near-) real-time analysis
− Privacy and security are important issues
− Data can be biased- we need to know our data!
5

6. Data Lifecycle
6
Source: The IET Technical Report, Digital Technology Adoption in the Smart Built Environment: Challenges and opportunities of
data driven systems for building, community and city-scale applications,
http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

7. Device/Data interoperability
7
The slide adapted from the IoT talk given by Jan Holler of Ericsson at IoT Week 2015 in Lisbon.

8. W3C semantic sensor network ontology (SSNO)
http://www.w3.org/2005/Incubator/ssn/ssnx/ssn
M. Compton, P. Barnaghi, L. Bermudez, et al, "The SSN Ontology of the W3C Semantic Sensor Network Incubator Group", Journal of Web Semantics,
2012.
8

9. Semantic annotation and vocabularies
P. Barnaghi, M. Presser, K. Moessner, "Publishing Linked Sensor Data", in Proc. of the 3rd Int. Workshop on
Semantic Sensor Networks (SSN), ISWC2010, 2010.
9

10. IoTLite Ontology
10http://iot.ee.surrey.ac.uk/fiware/ontologies/iot-lite

11. Search on the Internet/Web in the early days
1111

12. A discovery engine for the IoT
12A. HosseiniTabatabaie, P. Barnaghi, C. Wang, L. Dong, R. Tafazolli, "Method and Apparatus for Scalable Data Discovery in IoT Systems”, US
Patents, CNV12174, May 2014.
Let’s assume that attribute x has an
alphabet Ax ={ax1,…,axs}. Query for
a data item (q) that is described
with attributes x, y and z, is then
represented as q={x=axk & y=ayl &
z=azm}
The average ratio of matching
processes that are required to
resolve this query at n:

13. IoT environments are usually dynamic and (near-)
real-time
13
Off-line Data analytics
Data analytics in dynamic environments
Image sources: ABC Australia and 2dolphins.com

14. Creating Patterns-
Adaptive sensor SAX
14
F. Ganz, P. Barnaghi, F. Carrez, "Information Abstraction for Heterogeneous Real World Internet Data”, IEEE Sensors Journal, 2013.

15. From SAX patterns to events/occurrences
15
F. Ganz, P. Barnaghi, F. Carrez, "Automated Semantic Knowledge Acquisition from Sensor Data", IEEE Systems Journal, 2014.

16. From patterns/events to a situation ontology
16
F. Ganz, P. Barnaghi, F. Carrez, "Automated Semantic Knowledge Acquisition from Sensor Data", IEEE Systems Journal, 2014.

17. Learning ontology from sensory data
17

18. KAT- Knowledge Acquisition Toolkit
http://kat.ee.surrey.ac.uk/
F. Ganz, D. Puschmann, P. Barnaghi, F. Carrez, "A Practical Evaluation of Information Processing and Abstraction Techniques for the Internet of
Things", IEEE Internet of Things Journal, 2015.
18

19. Real world data
19

20. Analysing social streams
20
With

21. City event extraction from social streams
21
Tweets from a city
POS
Tagging
Hybrid NER+
Event term
extraction
GeohashingGeohashing
Temporal
Estimation
Temporal
Estimation
Impact
Assessment
Impact
Assessment
Event
Aggregation
Event
AggregationOSM LocationsOSM Locations SCRIBE ontologySCRIBE ontology
511.org hierarchy511.org hierarchy
City Event ExtractionCity Event Annotation
P. Anantharam, P. Barnaghi, K. Thirunarayan, A.P. Sheth, "Extracting City Traffic Events from Social Streams", ACM Trans. on Intelligent
Systems and Technology, 2015.
Collaboration with Kno.e.sis, Wright State University

22. CRF formalisation – for annotation
22
A General CRF Model

23. Geohashing
23
0.6 miles
Max-lat
Min-lat
Min-long
Max-long
0.38 miles
37.7545166015625, -122.40966796875
37.7490234375, -122.40966796875
37.7545166015625, -122.420654296875
37.7490234375, -122.420654296875
4
37.74933, -122.4106711
Hierarchical spatial structure of geohash for
representing locations with variable precision.
Here the location string is 5H34
0 1 2 3 4 5 6
7 8 9 B C D E
F G H I J K L
0 1
7
2 3 4
5 6 8 9
0 1 2 3 4
5 6 7
0 1 2
3 4 5
6 7 8

24. Automated creation of training data
24
Evaluation over 500 randomly chosen tweets from around 8,000 annotated tweets

25. Extracted events and the ground truth
25Open source software: https://osf.io/b4q2t/

26. Social media analysis
26
http://iot.ee.surrey.ac.uk/citypulse-social/

27. Not so good examples
27

28. CityPulse demo
28

29. CityPulse: live events from the city of Aarhus
29
http://www.ict-citypulse.eu/
29

30. Deep IoT
30

31. In summary
31
“The ultimate goal is transforming the raw data
to insights and actionable knowledge and/or
creating effective representation forms for
machines and also human users and creating
automation.”
This usually requires data from multiple sources,
(near-) real time analytics and visualisation
and/or semantic representations.

32. 32
“Data will come from various source and from
different platforms and various systems.”
This requires an ecosystem of IoT systems with
several backend support components (e.g.
pub/sub, storage, discovery, and access services).
Semantic interoperability is also a key
requirement.

33. IoT discovery engines?
33
“Working across different systems and various
platforms is a key requirement. Internet search
engines work very well with textual data, but IoT
data comes in various forms and often as
streams.”
This requires an ecosystem of IoT systems with
several backend support components (e.g.
pub/sub, storage, discovery, and access services).

34. IoT discovery engines?
34
“ To make it more complex, IoT resources are
often mobile and/or transient. Quality and trust
(and obviously privacy) are among the other key
challenges”.
This requires efficient distributed index and
update mechanisms, quality-aware an resource-
aware selection and ranking, and privacy control
and preservation methods (and governance
models) .

35. Accessing IoT data
35
“ The internet/web norm (for now) is usually
searching for the data; the search engines are
usually information locators – return the link to
the information; IoT data access is more
opportunistic and context aware”.
This requires context-aware and opportunistic
push mechanism, dynamic device/resource
associations and (software-defined) data routing
networks.

36. Web search is already adapting this model
36
Image credits: the Economist

37. The future: borders will blend
37Source: IEEE Internet Computing, Special issue on Physical-Cyber-Social Computing

38. In conclusion
− IoT data analytics is different from common big data analytics.
− Data collection in the IoT comes at the cost of bandwidth, network,
energy and other resources.
− Data collection, delivery and processing is also depended on multiple
layers of the network.
− We need more resource-aware data analytics methods and cross-layer
optimisations (Deep IoT).
− The solutions should work across different systems and multiple platforms
(Ecosystem of systems).
− Data sources are more than physical (sensory) observation.
− The IoT requires integration and processing of physical-cyber-social data.
− The extracted insights and information should be converted to a feedback
and/or actionable information.
38

39. Smart city datasets
39
http://iot.ee.surrey.ac.uk:8080

40. IET sector briefing report
40
Available at: http://www.theiet.org/sectors/built-environment/resources/digital-technology.cfm

41. Q&A
− Thank you.
− EU FP7 CityPulse Project:
http://www.ict-citypulse.eu/
@pbarnaghi
p.barnaghi@surrey.ac.uk