Soren Auer (IAIS Fraunhofer) presentation for Big Data in Secure Societies Workshop -Brussels, 30th Septmebr 2015-

1. BIG DATA EUROPE
Integrating Big Data, Software & Communities for
Addressing Europe’s Societal Challenges

2. negative conotation
Big Data has often a
6-oct.-15www.big-data-europe.eu

3. Big Data in Marketing
6-oct.-15www.big-data-europe.eu

4. Big Data in Intelligence
6-oct.-15www.big-data-europe.eu

5. BigDataEurope aims to help maximizing
the societal value of Big Data
 Health, demographic change and wellbeing;
 Food security, sustainable agriculture and forestry, marine and
maritime and inland water research, and the Bioeconomy;
 Secure, clean and efficient energy;
 Smart, green and integrated transport;
 Climate action, environment, resource efficiency and raw
materials;
 Europe in a changing world - inclusive, innovative and
reflective societies;
 Secure societies - protecting freedom and security of Europe
and its citizens.
6-oct.-15www.big-data-europe.eu

6. The three Big Data „V“ – Variety
is often neglected
Quelle: Gesellschaft für Informatik

7. © Fraunhofer-Allianz Big Data 7
Proactive Maintenance at Rolls Royce
New Business Model integrating Sensor Data & Big
Data Analytics
Dr. Dirk Hecker
Condition Monitoring, Proactive maintenance, „Power-by-the-hour“,
as-a-service Business Model – payment modell by flight hours
Quelle: www.springboeck.ch/SR_Technics.htm
© Mark Hillary | Flickr

8. © Fraunhofer-Allianz Big Data 8
The rolling Smartphone
New Business Models for the Automotive Industry
with Data Value Chains
Dr. Dirk Hecker
Windshield wiper as rain sensors for micro wether prognosis
Automotive industry can become data provider for other industries
Quelle:GTÜ
Quelle:www.farming-simulator.com

9. © Fraunhofer-Allianz Big Data 9
Predictive Analytics
Dr. Dirk Hecker
From Business Intelligence to Big Data Analytics
Business Intelligence Monitoring Predictive Analytics
What happened
before?
What happens
now?
What will happen
soon?
What should
happen?
Prescriptive Analytics
„the last Mile“
“prescriptive analytics suggests decision options on
how to take advantage of a future opportunity”
Quelle: BMW Quelle: www.7-forum.com Quelle: BMW Quelle: Volvo

10. BigDataEurope Rationale
 Show societal value of Big Data
 Lower barrrier for using big data technologies
o Required effort and resources
o Limited data science skills
o Lack of Generic Architectures, components
 Help establishing cross-
lingual/organizational/domain Data Value Chains
o Multiple Data Sources
o Required: Integration, Harmonisation
6-oct.-15www.big-data-europe.eu

11. BigDataEurope: Objectives
6-oct.-15www.big-data-europe.eu
COORDINATION
Stakeholder Engagement
(Requirements Elicitation)
SUPPORT
Design, Realise, Evaluate
Big Data Aggregator
Platform
Create and Manage
Societal Big Data
Interest Groups
Cloud-deployment
ready
Big Data Aggregator
Platform
CSA
Measures
Results

12. Orthogonal Dimensions of Big Data Ecosystems
Generic Big Data Enabling Technologies
Data Value Chain
Data Generation
& Acquisition
Data Analysis &
Processing
Data Storage &
Curation
Data
Visualization &
Usage
Data-driven
Services
SocietalChallenges
DomainSpecificDataAssets&Technology
Healthcare
Food Security
Energy
Intelligent Transport
Climate & Environment
Inclusive & Reflective Societies
Secure Societies

13. Stakeholder Engagement Cycle

14. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Open-Source Technologies for Big Data Apps (small selection :-)
14

15. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Big Data - Technologies
Volume
VelocityVariety
Storm
15

16. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Groups of Technologies
Big Data
Technolog
ies
Data Storage
Technologies
Data
Processin
g
Workflow
Coordinati
on
Querying/
Processin
g
Search
Data
Export/
Import
Data
Analysis
Statistics
Text
Mining
16

17. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Big Data Requirements
Analysis of historical dta
 Millions of entries
 Varying analysis quesitions
 Years of input data
 => Big Data Batch Processing
Interactive analysis by online queries
 Thousands of users online
 Extremely fast response time
 Super high availability
 => Big Data Databases
Analysis of actual data with low latency in
"real-time"
 React to newest trends
 Low-Latency change detection
 Real-time online monitoring
 => Big Data Stream Processing
But how to put it together ?
17

18. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
The „traditional“ Hadoop Ecosystem + NoSQL
components
a Big Data Management System
ZooKeeper
askaban
Kafka
cassandra
voldemort
MongoDB
CouchDB
elastic search
solr
lucene

19. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Batch
Function
Speed
Function
Data Storage
pages with
postings
Batch View
Realtime
View
messagepassing
message passing
Application
Horizontal Scalability in the Lambda Architecture
19
> volume
> users
> users, volume
> velocity> volume, velocity

20. Blueprint of the Data Aggregator Platform
 Follows typical Lambda Architecture
 Integrated on top of existing Big Data distribution
 + Semantic Layer (Retaining Semantics using LD approach )
Batch Layer
Speed Layer
Data Storage
Real-time data &
Transactions …
Batch View
Real-time
View
messagepassing
message passing
Applications & Showcases
Real-time dashboards
Domain-specific BDE apps
Big Data Analytics
In-stream Mining
BDEPlatform&Intelligence
Input data
Stream
Spatial
Social
Statistical
Temporal
Transactional
Imagery

21. BDE Platform based on BigTop
Packaging Smoke testing Virtualization
Package RPMs and DEBs, so
that you can manage and
maintain your own cluster.
Integrated smoke testing
framework
Vagrant recipes, raw images,
and docker recipes for
deploying BigData
infrastructures from zero.
6-oct.-15www.big-data-europe.eu
+ Semantic Layer - Retaining Semantics using Linked Data

22. Data Aggregator Platform Challenges
 Ingest semantic (RDF) and non-semantic (CSV,
JSON, XML, …) data
o Integrate various mapping techniques (R2RML, CSV on
the Web, JSON-LD)
 preserve semantics, provenance and metadata in
Big Data processing chains
o Preserve URI/IRIs
o Preserve triples
 Exploit semantics for aggregations
6-oct.-15www.big-data-europe.eu

23. Current Activities – Year#1
 2015 BDE Societal Workshops (7) Planned
o Schedule on Website
 7 W3C Interest Groups set up: Please Join!
o SC1: HEALTH https://www.w3.org/community/bde-health/join
o SC2: FOOD & AGRICULTURE https://www.w3.org/community/bde-food/
o SC3: ENERGY https://www.w3.org/community/bde-energy/
o SC4: TRANSPORT https://www.w3.org/community/bde-transport/
o SC5: CLIMATE & ENVIRONMENT https://www.w3.org/community/bde-climate/
o SC6: SOCIETIES https://www.w3.org/community/bde-societies/
o SC7: SECURITY https://www.w3.org/community/bde-secure-societies/
www.big-data-europe.eu

24. BDE Partners

25. Sören Auer
Big Data Europe Coordinator
Fraunhofer IAIS & University of Bonn
auer@cs.uni-bonn.de
Thanks
6-oct.-15www.big-data-europe.eu

26. Energy/Climate Example: Greenshifting
6-oct.-15www.big-data-europe.eu