Workshop on FIWARE, Docker Virtualisation and Austrian Address Data released as public sector information

1. Johann Höchtl15. April 2016
Österreichische Adressdaten
auf FIWARE
FIWARE Workshop – FIWARE kennenlernen und nutzen
15 April 2016 – ATOS IT Solutions
Siemensstraße 92, 1210 Wien

2. 15. April 2016 2
Motivation
Image CC https://flic.kr/p/b3VC6X
1. What is FIWARE?
(and what is my money spent on?)
2. Save money by not sending it to
Heroku <INSERT_CLOUD_PROVIDER>
3. Learn sth. new
on a real world challenge

3. 15. April 2016 3

4. 15. April 2016 4
http://map.fiware.org/actors/smes

5. 15. April 2016 5

6. 15. April 2016 6

7. 15. April 2016 7
Image CC https://flic.kr/p/7ajM9d
How the EU and giants like
Telefonica or IBM are wasting
€100 million
and entrepreneurs’ time
https://novobrief.com/fiware-european-union-telefonica-ibm-waste-of-time/

8. 15. April 2016 8
FIWARE Facts & Funding
(http://de.slideshare.net/chicco785/fiware-primer-learn-fiware-in-60-minutes/10, November 2015)
Budget for FIWARE/FIWARE Lab (phases 1 & 2)
126 M€ = (89 + 37) M€
countries
270 150
companiespartners
400 M€ = (300 + 100) M€
Budget (RTD + funding for entrepreneurs)
>1.000
SMEs and start-ups
27

9. 15. April 2016 9
What is FIWARE?
(Credit http://tinyurl.com/fiware-open-ecosystem )
Open Standard
Platform
ecosystem
open globalsustainable
OpenStack-based Cloud &
Library of Generic Enablers
support developers to
create Internet
applications and startups
based on FIWARE
establish worldwide links
that help FIWARE to
thrive beyond European
borders

10. 15. April 2016 10
FIWARE Generic Enablers
Publish/Subscribe Context Broker -
Orion Context Broker
Complex Event Processing (CEP) -
Proactive Technology Online
Identity Management - KeyRock
PEP Proxy – Use the FIWARE
account for authentication
Docker

11. 15. April 2016 11
Openstack – Overview & FIWARE Usage
NOVA
IMAGE
SERVICE
OBJECT
STORE
VOLUME
SERVICE
IDENTITY
SERVICE
GLANCE SWIFT CINDER
KEYSTONE
OPST0005
COMPUTE NETWORKING
NEUTRON HEAT
Orchestration
CEILOMETER
Telemetry
DASHBOARD
HORIZON

12. 15. April 2016 12
Example: Smart City Domain
(Credit: http://de.slideshare.net/flopezaguilar/introduction-to-fiware-open-ecosystem/44)
CKAN
Big Data
Context Broker
Accounting&Payment&
Billing
IDM&Auth
Short-
term
historic
data
BigData
Processing
Data
Quering/Actio
n,
Publish/Subsc
r
Open Data
publishing
Real-time
processing
BI
ETL
RULES
DEFINITION
TOOL
OPEN DATA
PORTALS
Service
orchestra
tor
Context
Adapters
CEP
IoT Backend
Device
Managementmeasures
/
comman
ds
IoT Broker & Config
Management
(from sensors to things)
IoT/Sen
sor
Open
Data
actuato
rs
Media
streams
Real
Time
Media
Stream
Processi
ng
City
Services
OPERATIONA
L
DASHBOARD
KPI GOVERNANCE
GIS
Inventory
Specific Enablers
Generic Enablers

13. 15. April 2016 13
FIWARE GE Future Scenario
FIWARE Catalogue
creation of
dedicated GE
instances
Use of Global
shared GE instance
FIWARE provider A
Applications
FIWARE provider B
Applications

14. 15. April 2016 14
Getting go - Register
https://account.lab.fiware.org/
1 get account
2 get (almost) nothing
https://ask.fiware.org/question/6/do-i-need-to-be-part-of-an-accelerator-project-in-order-to-use-

15. 15. April 2016 15
Apply for Community Account

16. 15. April 2016 16
Computation Flavours
https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/FIWARE_Lab:_Upgrade_to_Community_Account#Quotas_and_Flavours
Flavour ID Flavour Name Memory User Disk Virtual CPUs
1 m1.tiny 512mb 1gb 1
2 m1.small 2048mb 20gb 1
3 m1.medium 4096mb 40gb 2
4 m1.large 8192mb 80gb 4

17. 15. April 2016 17
FIWARE Lab Nodes

18. 15. April 2016 18
Apply for community account
Description: Dockerify Postgresql
Status: To Do
Components: (Others)
User full name: jhoechtl
User account to be upgraded: johann.hoechtl@gmail.com
Accelerator submission name:
Preferred FIWARE Lab Node: Trento
Number of VMs: 1.0
# public IPs: 1.0
total # vCPUs: 2.0
total RAM: 4.0
total harddisk: 40.0
Object Storage:
5.12. 7.12.
In order to obtain a "Community account"
on FIWARE Lab, the applicant has to
described the experimentation he/she
plans to do and how it is related with the
FIWARE ecosystem. Based on this
description the community account is
granted or not (mainly it depends on the
innovative idea proposed and how this
idea leverages on FIWARE).
9.12.
Sent description
14.12.
Nothing happened; Inquiry
15.12.
Scheduled

19. 15. April 2016 19
Apply for community account
15.12.
On Trento Node in order to get a floating
IP, you should provide us a copy of the
identity card/passport of the main user
or responsible of the project.
Please contact us at: support-
xifi@trentinonetwork.it to step forward
with the procedure.
17.12.
FloatingIP has also been
released after identity
verification procedure
completed.
User has been notified via email
with details.

20. 15. April 2016 20
Docker on FIWARE Lab Instance
https://myprivate42.wordpress.com/2016/02/26/docker-on-fiware/
 OpenStack-based IaaS to provide computing and storage capacity across multiple
data centers
 Docker based PaaS to deploy and compose scalable applications

21. 15. April 2016 21
Digression: Docker Why and What
(Credit http://de.slideshare.net/dotCloud/docker-intro-november)
App
A
Hypervisor (Type 2)
Host OS
Server
Guest
OS
Bins/
Libs
App
A’
Guest
OS
Bins/
Libs
App
B
Guest
OS
Bins/
Libs
VM
Guest
OS
Guest
OS
A
pp
A’
Docker
Host OS
Server
Bins/Lib
s
A
pp
A
Bins/Libs
A
pp
B
A
pp
B’
A
pp
B’
A
pp
B’
Container
FIWARE (Openstack) Instance
Containers are isolated, but share OS
and, where appropriate, bins/libraries
Faster deployment, less overhead,
easier migration

22. 15. April 2016 22
Infrastructure Support
Digression: Docker Why and What
(Credit http://de.slideshare.net/dotCloud/docker-intro-november)
Tool Supporthttps://docs.docker.com/machine/get-started-cloud/
Amazon Web Services, Digital Ocean,
Google Compute Engine, IBM Softlayer,
Microsoft Azure, Microsoft Hyper-V,
OpenStack, Oracle VirtualBox, Rackspace,
VMware Fusion, VMware vCloud Air,
VMware vSphere, exoscale
1. Abstract Infrastructure from Platform
2. Continue working in your known environment
3. Prevent vendor lock-in: Standardized container format OCI and supervision
http://de.slideshare.net/Docker/docker-birthday-3-intro-to-docker-slides

23. 15. April 2016 23
Open Container Initiative
Alternatives to Docker
+

24. 15. April 2016 24
FIWARE Openstack and Docker
https://wiki.openstack.org/wiki/Docker
https://docs.docker.com/machine/drivers/openstack/

25. 15. April 2016 25
Functional overlap Openstack vs. Docker
Openstack (FIWARE) Docker
Deployment nova boot <instance> docker run <container>
Orchestration Heat docker swarm; Kubernetes
Storage Volume (Cinder) & Objects (Swift) Data-only container
Logical service grouping Blueprints Docker compose
Service Orchestration with Kubernetes

26. 15. April 2016 26
BEV Address Data
Excerpt of Address register (Adressregister), Go To Link
Up to now, two releases: 15.07.2015, 08.10.2015
8 CSV files, ~ 550MB
File Size Line count
ADRESSE.csv 288223907 2335801
ADRESSE_GST.csv 72282856 2503351
GEBAEUDE.csv 199002301 2336346
GEBAEUDE_FUNKTION.csv 11031600 344737
GEMEINDE.csv 51779 2101
ORTSCHAFT.csv 514326 17259
STRASSE.csv 5134464 131081
ZAEHLSPRENGEL.csv 314096 8826

27. 15. April 2016 27
BEV Address Data
Gemeinde
(GKZ)
● Gemeindename
Ortschaft
(OKZ)
● Ortsname
Strasse
(SKZ)
● Strassenname
Adresse
(ADRCD)
● PLZ
● Hausnrtext
● Hofname
● RW, HW
● Bestimmungsart
GEBAEUDE
(ADRCD, SUBCD)
Municipality Names
Street names
Districts and Sub-entities
ZIP-Codes
Geo-Codes of Addresses and Buildings

28. 15. April 2016 28
BEV Address Data
1. Analyse Data
2. Import Data into relational Database
3. Provide full text search via Web interface
4. URIfy Address Data – provide a public unique identifier for each address in
object

29. 15. April 2016 29
BEV Address Data
Observations …
● RW and HW are in Datum Austria → not suitable for general mapping on
Google Maps or Openstreetmap
● Irregular naming of city districts:
"1010";"Wien";"Wien,Innere Stadt";"Kärntner Straße";"90001";"17223"
"8043";"Graz";"Graz,11.Bez.:Mariatrost";"Mariatroster Straße"
"9020";"Klagenfurt am Wörthersee";"Klagenfurt,07.Bez.:Viktringer Vorstadt";"Bahnhofstraße"
● Abbreviations
geg., Bez., Parz.

30. 15. April 2016 30
BEV Address Data
Observations and countermeasures
● RW and HW are in Datum Austria → not suitable for general mapping on
Google Maps or Openstreetmap
→ Use PostGIS to convert coordinates:
UPDATE ADRESSE SET MGIAUSTRIAGK = ST_SetSRID(ST_MakePoint(RW, HW), epsg);
ALTER TABLE ADRESSE ADD COLUMN LATLONG geometry(POINT);
UPDATE ADRESSE SET LATLONG = ST_Transform(MGIAUSTRIAGK, 4326);
● Siemensstraße 92?
RW: 6967.7 HW: 347724.57 POINT(16.4219750234186 48.2678291034249)
http://www.openstreetmap.org/?mlat=48.2678291034249&mlon=16.4219750234186

31. 15. April 2016 31
BEV Address Data - Observations
● Irregular naming of city districts & abbreviations
→ Configure full text search with synonym and thesaurus dictionary:
Innere:1 01.bez erster innere
13.bez: dreizehnter 13 13.bez
#Klagenfurt
Viktring:13 12.bez dreizehnter Viktring
● Query “Wien, erster Bezirk”
→ "1010";"Wien";"Wien,Innere Stadt";"An der Hülben"
● Query “Wien, innere stadt”
→ "1010";"Wien";"Wien,Innere Stadt";"An der Hülben"
● Query “Graz, erster bezirk”
→ "8010";"Graz";"Graz,01.Bez.:Innere Stadt";"Hans-Sachs-Gasse"

32. 15. April 2016 32
BEV Address Data – Import Data into DB
● There is no relational data base system available on FIWARE
→ Solution: Use Docker to provision open source database PostGIS

33. 15. April 2016 33
Problems encountered
● Official documentation didn't match reality:
– http://simple-docker-hosting-on-fiware-cloud.readthedocs.org/en/latest/manuals/install/
1. Custom security groups didn't work out of the box
2. FIWARE Operating system images required modified options for
provisioning
3. Ports for Docker orchestration and management on FIWARE
not open by default
→ local FIWARE node support necessary
FIWARE email support was generally responsive and helpful

34. 15. April 2016 34
BEVADDRESS on FIWARE – Web Interface
● Full text search for addresses with auto completion
● Return unique URI for address
● Ability to display on Google Maps / OSM
● Search Address within certain radius after click on map
● Analysis: Longest street in Austria, Streets spanning districts, streets with
“funny” names, etc., etc.,
● Also see https://opendata.host/de (requires API-key)

35. 15. April 2016 35
BEVADDRESS on FIWARE – You are invited!
Modified PostGIS Database optimized for BEVADDRESS full text search
https://github.com/the42/bevaddress-dataload
Scripts to setup PostGIS and load address data
https://github.com/the42/bevaddress-dataload
User Interface
TBD → Your help wanted!

36. 15. April 2016 36
FIWARE Critique
Documentation is spread all over
http://help.lab.fiware.org/
11. - 13. 4. 2016

37. 15. April 2016 37
Critique
Documentation is spread all over
http://de.slideshare.net/fi-ware
http://edu.fiware.org/
https://forge.fiware.org/plugins/mediawiki/wiki/fiware/index.php/Main_Page
https://readthedocs.org/search/?q=FIWARE
http://catalogue.fiware.org/
https://ask.fiware.org/questions/
http://stackoverflow.com/questions/tagged/fiware

38. 15. April 2016 38
Critique
Lacking middleware

39. 15. April 2016 39
Critique
Lacking middleware
How to get there?

40. 15. April 2016 40
Critique
Innovation friendly?
Lego advertisment 1981 Lego advertisment 2013

41. 15. April 2016 41
Reliability and Issues
Trento Node

42. 15. April 2016 42
Reliability and Issues –
Events in the life of a FIWARE app
26.11.2015 The FIWARE Lab will schedule a maintenance time slot on December,
10th from 09:00 till 11:00 am
04.12.2015 Cygnus 0.11.0 has been released. As usual, it is highly recommended
you upgrade you current installation.
01.02.2016 A new Orion Context Broker release (0.27.0) has just been released,
18.02.2016 We are experimenting some network problem in Spain2 and probably
you cannot access to your resources
29.02.2016 A new Orion Context Broker release (0.28.0) has just been released,
03.03.2016 Tomorrow (Fridata, March 4) the Cosmos service in FIWARE Lab will
be affected by a maintenance task; certain hardware must be replaced
in the hosting machines
10.03.2016 The Budapest2 node is experiencing some issues with the blade
system.
29.03.2016 A new Orion Context Broker release (1.0.0) has just been released
11.04.2016 next Monday, 11 April, a maintenance task has been scheduled in the

43. 15. April 2016 43
FIWARE Future
https://ask.fiware.org/question/460/remaining-lifetime-of-fiware-lab-and-commercial-instances/
How much longer will FIWARE Lab
exist (and be accessible to us)?
Given the above, FIWARE Lab will be
operational following the interests
of its users and stakholders.
Current plans say at least end of
2020.

44. 15. April 2016 44
Findings
● Overall stability not yet enough to base a business model on
● Confusing accumulation of components, startup accelerator landing pages,
documentation, infrastructure administration
● FIWARE node communication helpful and timely

45. 15. April 2016 45
FIWARE is the European cloud
Time to start experimenting
CC https://flic.kr/p/qVdaZS

46. 47
Donau-Universität Krems.
Die Universität für Weiterbildung.
Johann Höchtl
Center for E-Governance
Johann.hoechtl@donau-uni.ac.at
@myprivate42
at.linkedin.com/in/johannhoechtl
github.com/the42
myprivate42.wordpress.com/
CC-BY 4.0