This document describes a pilot project that aims to create an online dashboard of municipal budget and spending data from multiple European cities. The project will harvest, link, analyze and visualize open budget data to make it more useful for citizens, researchers and decision makers. Data will be integrated from Athens, Thessaloniki and Kalamaria in Greece initially, and potentially Vienna, Linz and Barcelona. The technical architecture uses Apache tools like Flume, Kafka and Spark to ingest, store and analyze the heterogeneous data sources in real-time. The goal is to evaluate how a big data approach can provide new insights into public finances from an integrated, multilingual and longitudinal perspective.

1. BIG DATA EUROPE
SC6: EUROPE IN A CHANGING WORLD - INCLUSIVE, INNOVATIVE
AND REFLECTIVE SOCIETIES
PILOT SC6: CITIZEN BUDGET ON MUNICIPAL LEVEL
BDE SC6 WORKSHOP AMSTERDAM, The Netherlands, 11.09.2017
M KALTENBÖCK (SWC)
HTTP://WWW.BIG-DATA-EUROPE.EU/
Integrating Big Data, Software & Communities for Addressing
Europe’s Societal Challenges

2. SC6 in Big Data Europe
◎ Europe in a changing world - inclusive,
innovative and reflective societies
◎ Social Sciences
◎ Smart Statistics
◎ (Digital) Humanities
◎ Digital (Research) Archives
27-sept.-16www.big-data-europe.eu

3. Integrating Big Data, Software & Communities for Addressing
Europe’s Societal Challenges
BIG DATA EUROPE
PILOT SC6: CITIZEN BUDGET ON MUNICIPAL LEVEL
STATUS & OUTLOOK

4. SC6 Pilot: Idea & Objectives
◎ Create an online Dashboard on Economic Data
o Harvest data from several sources in diff. formats
o Normalise the data (RDF)
o Link & map the data (attributes, structure, languages)
o Analyse the data (comparisons, predictions etc.)
o Visualise the analysis on an online dashboard
including help & infos to understand data & analysis
o Provide raw data (for further use as open data)
27-sept.-16www.big-data-europe.eu

5. SC6 Pilot: Idea & Objectives
State of the Art:
o Budget: the most important document of public policy
o Budget execution affects everyday lives
o Citizens are more involved in city level activities
Objective:
Can we make budgets more useful for citizens,
researchers and decision makers?
27-sept.-16

6. 3 BDE partners working together on the SC6 Pilot
Big Data Europe
SC6 Pilot core team: Ivana Versic (Cessda), Michalis Vafopoulos (NCSR-D),
Martin Kaltenböck (SWC), Hossein Abroshan (Cessda),
Jürgen Jakobitsch (SWC), Victor Mireles-Chavez (SWC).
Cooperation with 2 projects: YourDataStories and OpenBudget
SC6 Pilot Partners

7. Data used / produced in Pilot
Budget Data & Budget Execution Data
◎ Municipality of Athens, Greece
o Description: budget execution data in detail (account level)
o Frequency: daily
o Ownership: open
o Format: API
◎ Municipality of Thessaloniki, Greece
o Description: budget execution data in detail
o Frequency: daily
o Ownership: open
o Format: csv, xls (files for download provided)
◎ Municipality of Kalamaria, Greece
o Description: budget execution data in detail
o Frequency: daily
o Ownership: open
o Format: csv, xls (files for download provided)
27-sept.-16www.big-data-europe.eu
Additional Open Data
o Description: economic taxonomies etc.
o Ownership: open
o Format: RDF (skos, owl), other
o E.g. COFOG (UN Classification)
o AND: BDE team modeled data (financial ratios,
budget items etc al)
Additional data to be integrated
o Budget data from several European municipalities
o In evaluation:
❖ Vienna (AT)
❖ Linz (AT)
❖ Barcelona (ES)

8. 4 Vs of Big Data in SC6 Pilot
◎ Variety: requirement based on the harvesting of budget data and budget
execution data from several sources, available in different structures and
formats.
◎ Volume: requirement regarding the growing amount of open budget data
available as well as of budget execution data
◎ Velocity: requirements regarding budget execution data that is provided on
continuous basis by the publisher (daily, weekly, monthly).
◎ Veracity: Veracity refers to the biases, noise and abnormality in data. Even
within the same country there are differences on the published data because
often it: a) comes from different systems or b) public accounting standards are
not enforced absolutely uniformly (across e.g. different municipal departments)
www.big-data-europe.eu

9. SC6 Pilot - Architecture
27-sept.-16www.big-data-europe.eu

10. SC6 Pilot: Technical Components
◎ Apache Flume, https://flume.apache.org/ (data ingestion)
◎ Apache Kafka, http://kafka.apache.org (messaging service)
◎ Apache Spark, http://spark.apache.org (distributed analysis, transformation)
◎ Apache HDFS, http://hadoop.apache.org (raw data storage)
◎ SWCs’ PoolParty Semantic Suite, http://poolparty.biz (data consolidation, curation, mapping)
◎ SWCs’ PoolParty Graph Search, link (search, browse & visualise data)
◎ OpenLink s’ Virtuoso, http://virtuoso.openlinksw.com (triple store: data storage, data endpoint / API)
◎ Apache HTTP, http://httpd.apache.org (linked data serving)
◎ Apache Avro, http://avro.apache.org/docs/current/ (intermediate data schema)
◎ D3 JS Library, https://d3js.org/ (visualisation of RDF data using SPARQL queries)
27-sept.-16www.big-data-europe.eu

11. In progress
◎ Invite municipalities to evaluate and use the system
◎ Invite community (open data, data community, BDE community, W3C)
◎ Evaluate within / together with the 3 participating projects (BDE, DataStories, OpenBudget)
◎ BDE SC6 workshop in Amsterdam, 11.09. 2017 PLUS final webinar (hangout) on 02.11.2017
Additional evaluation – tests over time with
◎ a growing amount of data
◎ a growing number of different sources & formats docked onto the system
◎ Improve and expand mappings & linking (schema and data)
◎ additional analytics in place
27-sept.-16www.big-data-europe.eu
SC6 Pilot: Pilot (Evaluation) Activities

12. SC6 Pilot: Status & Outlook
BDE SC6 Pilot - next steps planned
◎ Update Pilot on latest (and final) BDI version (and maintain the Pilot)
◎ Finalise data ingestion from additional (in clarification: Austrian & Spanish) municipalities
◎ Expand data dashboard and data API (endpoint)
◎ Improve performance and stability of the BDI based Pilot
◎ Refine on the basis of users evaluation and feedback
◎ Prepare for handover / operation after BDE project duration
◎ Workshop (11.9.) and final HangOut SC6 (2.11.2017)
www.big-data-europe.eu

13. SC6 Pilot: Short Demo
27-sept.-16www.big-data-europe.eu https://bde.poolparty.biz/GraphSearchSC6/

14. Contacts:
◎ CESSDA, http://cessda.net/
Ivana Ilijasic Versic, ivana.versic@cessda.net
Hossein Abroshan, hossein.abroshan@cessda.net
◎ NCSR-D, http://www.demokritos.gr/?lang=en
Michalis Vafopoulos, vafopoulos@gmail.com
◎ Semantic Web Company (SWC), http://www.semantic-web.at
Martin Kaltenböck, m.kaltenboeck@semantic-web.at
Jürgen Jakobitsch, j.jakobitsch@semantic-web.at
Victor Mireles-Chavez, victor.mireles-chavez@semantic-web.com
27-sept.-16www.big-data-europe.eu