This document discusses challenges and opportunities around working with real-world data. It notes that while data is plentiful, real-world data is difficult to obtain due to issues like data silos and privacy concerns. It also discusses problems with data interoperability, quality, reliability, and needing more than just analytics to gain insights. The document advocates for linked open data streams with metadata and scalable analytics tools combined with domain knowledge to create actionable knowledge from real-world data. It concludes by listing challenges and opportunities in providing infrastructure, publishing and analyzing heterogeneous and private data at scale.

1. Working with real data
1
Payam Barnaghi
Centre for Communication Systems Research (CCSR)
Faculty of Engineering and Physical Sciences
University of Surrey
Guildford, United Kingdom

2. 2
Things, Data, and lots of it
image courtesy: Smarter Data - I.03_C by Gwen Vanhee

3. Data is not what we want or is it?

4. What we need are insights
and actionable-knowledge

5. Diffusion of innovation
image source: Wikipedia
IoT

6. Problem #1
Data: We seem to have lots of it…
Real World Data: it is always difficult to get
(silos, format, privacy, business interests or
lack of interest!...)

7. Problem #2
Data: interoperability and metadata
frameworks…
Real World Data: there are solutions for
service based (Restful) access, meta-
data/semantic representation frameworks
(W3C SSN, HyperCat,…) but none of them
are widely adapted.

8. Problem #3
Data: quality, reliability…
Real World Data: data can be noisy, crowed
source data can be inaccurate,
contradictory, delay in accessing/processing
the data…

9. Problem #4
Data: having too much data and using
analytics tools alone won’t solve the
problem…
Real World Data: in addition to the HPC
issues, we need new methods/solutions that
can provide real-time analysis of dynamic,
variable quality and multi-modal streams…

10. Problem #5
Data: abstraction, discovering the
associations…
Real World Data: co-occurrence vs.
causation; we need hypothesis, background
knowledge,…
After all data is not what we are really
after…

11. We need more linked open data

12. (near) real-time
linked open data
Streams
Sometimes it’s even better if we have:

13. (near) real-time
linked open data
Streams
+
meta-data (semantic annotations)
+
Adaptable and scalable analytics tools
+
Sufficient background knowledge
or even better than that if we have:

14. Data analytics
14
Data:
DataData
Domain
Knowledge
Domain
Knowledge
Social
systems
Social
systems
InteractionsInteractionsOpen
Interfaces
Open
Interfaces
Ambient
Intelligence
Ambient
IntelligenceQuality and
Trust
Quality and
Trust
Privacy and
Security
Privacy and
Security
Open DataOpen Data

15. 15
Challenges and opportunities
− Providing infrastructure
− Publishing, sharing, and access solutions on a global scale
− Heterogeneity and interoperability at different layers
− Indexing, query and discovery (data and resources)
− Aggregation, integration and fusion
− Trust, privacy and security
− Data analytics and creating actionable knowledge
− Integration into services and applications in e-health, the public
sector, retail, manufacturing and personalised apps.
− Mobile apps, location-based services, monitoring control etc.
− New business models

16. − Thank you.
− EU FP7 CityPulse Project:
http://www.ict-citypulse.eu/
@ictcitypulse
p.barnaghi@surrey.ac.uk