About opendata2017

Lorenzino Vaccari 14/06/2017
Open Data seminar
(Please, interrupt me and make questions!)
1
lorenzino.vaccari@gmail.com
lorenzino.vaccari@ec.europa.eu
Lorenzino Vaccari
Seminar at POLIMI@Lecco

About me

14/06/2017Lorenzino Vaccari
Part 1: for data prosumers
• What are Open Data useful for?
• What is Open Data?
• Why are Open Data useful?
• How is Open Data related to Open
Government Data and Big Data?
• Using Open Data: a practical example
3

Open Source & Open Data together to tackle with
humanitarian projects and economic development
4
HOT: Humanitarian OpenStreetMap Team
http://hot.openstreetmap.org

Open Source & Open Data together to tackle with
humanitarian projects (before/ after)
5
HOT: Humanitarian OpenStreetMap Team
http://hot.openstreetmap.org

Cars in trentino
7

http://www.webmapp.it/mappe/dolomiti/

http://content.stamen.com/files/cartography/index_watercolor.html

Map of the pianos
10
https://github.com/brunetton/OpenPianosMap

Open Data blankets
11
https://github.com/brunetton/OpenPianosMap

Open Data Shoes
12
http://in2.ccio.co/K2/LA/G/218143175671272930BwpG7bSyc.jpg

What is Open Data?
13

“is data that can be freely used, reused and
redistributed by anyone – subject only, at most, to
the requirement to attribute and sharealike.” *
*(Source: )
http://opendatahandbook.org/guide/en/what-is-open-data/
14

“Open” data
15
• Open License
• Free
• Open Access, e.g.:
• No registration
• No co-authorship
• Direct access (no services)
• ….
https://unsplash.com/@ryanmoreno

• use
• reuse
• redistribution
• commercial reuse
• derivative works
BUT, may require:
• attribution
• share alike
J. Gray (OKF): http://www.slideshare.net/jwyg/open-government-data-what-why-how
16
“open” =

Open License
17
• A license should be compatible with other
open licenses.
• A license is open if its terms satisfy the
following conditions...
https://unsplash.com/@rzunikoff

Open license:
Required Permissions
18
The license must irrevocably permit (or allow) the following:
Use: The license must allow free use of the licensed work.
Redistribution: The license must allow redistribution of the licensed work, including sale,
whether on its own or as part of a collection made from works from different sources.
Modification: The license must allow the creation of derivatives of the licensed work and allow
the distribution of such derivatives under the same terms of the original licensed work.
Separation: The license must allow any part of the work to be freely used, distributed, or
modified separately from any other part of the work or from any collection of works in which it
was originally distributed. All parties who receive any distribution of any part of a work within
the terms of the original licenseshould have the same rights as those that are granted in
conjunction with the original work.
Compilation: The license must allow the licensed work to be distributed along with other
distinct works without placing restrictions on these other works.
Non-discrimination: The license must not discriminate against any person or group.
Propagation: The rights attached to the work must apply to all to whom it is redistributed
without the need to agree to any additional legal terms.
Application to Any Purpose: The license must allow use, redistribution, modification, and
compilation for any purpose. The license must not restrict anyone from making use of the work
in a specific field of endeavor.
No Charge: The license must not impose any fee arrangement, royalty, or other compensation
or monetary remuneration as part of its conditions.
http://opendefinition.org/od/2.1/en/

Open license:
Acceptable Conditions
19
The license must not limit, make uncertain, or otherwise diminish the permissions
required in Section 2.1 except by the following allowable conditions:
Attribution: The license may require distributions of the work to include attribution of
contributors, rights holders, sponsors, and creators as long as any such prescriptions are not
onerous.
Integrity: The license may require that modified versions of a licensed work carry a different
name or version number from the original work or otherwise indicate what changes have been
made.
Share-alike: The license may require distributions of the work to remain under the same license
or a similar license.
Notice: The license may require retention of copyright notices and identification of the license.
Source: The license may require that anyone distributing the work provide recipients with access
to the preferred form for making modifications.
Technical Restriction Prohibition: The license may require that distributions of the work remain
free of any technical measures that would restrict the exercise of otherwise allowed rights.
Non-aggression: The license may require modifiers to grant the public additional permissions
(for example, patent licenses) as required for exercise of the rights allowed by the license. The
license may also condition permissions on not aggressing against licensees with respect to
exercising any allowed right (again, for example, patent litigation).
http://opendefinition.org/od/2.1/en/

Open “Data”
20
Best practices:
• Primary source
• Timely
• Open format
• Updated and complete
• Machine readable
• ...

Data, document, service
21 http://data.jrc.ec.europa.eu/dataset/2a2c2db3-dfc5-4069-989a-30716a2a4152

Maurizio Napolitano: http://www.youtube.com/watch?v=YlkjrVAW43Q
Primary
source
22

Open Format & Machine Readable
23
http://5stardata.info/en/

https://upload.wikimedia.org/wikipedia/commons/7/79/14LaAc_periodic_table_IIb.jpg
“It’s great to have the
data accessible on the Web
under an open license (such
as PDDL, ODC-by or CC0),
however, the data is
locked-up in a document.
Other than writing a
custom scraper, it’s hard to
get the data out of the
document.”
make your stuff available on the Web
(whatever format) under an open license

“Splendid! The data is accessible on the
Web in a structured way (that is,
machine-readable), however, the data is still
locked-up in a document. To get the data
out of the document you depend on
proprietary software.”
make it available as structured data
(e.g., Excel instead of image scan of
a table)

DateTime,MC
2016-01-01 00:00:00.000,58.808331
2016-01-01 00:10:00.000,59.374001
2016-01-01 00:20:00.000,58.720833
2016-01-01 00:30:00.000,57.98
2016-01-01 00:40:00.000,57.606003
2016-01-01 00:50:00.000,56.762001
2016-01-01 01:00:00.000,55.659184
2016-01-01 01:10:00.000,54.94286
2016-01-01 01:20:00.000,54.263268
2016-01-01 01:30:00.000,52.922451
2016-01-01 01:40:00.000,53.167347
2016-01-01 01:50:00.000,54.807999
2016-01-01 02:00:00.000,57.063263
2016-01-01 02:10:00.000,58.257141
2016-01-01 02:20:00.000,58.035999
2016-01-01 02:30:00.000,57.861225
2016-01-01 02:40:00.000,57.07143
2016-01-01 02:50:00.000,56.338776
2016-01-01 03:00:00.000,55.452
….
http://data.jrc.ec.europa.eu/dataset/jrc-abcis-ap-pm10mc-2016
“Excellent! The data is not only
available via the Web but now
everyone can use the data easily. On
the other hand, it’s still data on the
Web and not data in the Web.”
make it available in a
non-proprietary open format
(e.g., CSV as well as of Excel)

https://data.europa.eu/euodp/en/data/dataset/jrc-names
“Wonderful! Now it’s
data in the Web. The
(most important) data
items have a URI and can
be shared on the Web. A
native way to represent
the data is using RDF,
however other formats
such as Atom can be
converted/mapped, if
required.”
use URIs to denote things,
so that people can point at
your stuff

“Brilliant!
Now it’s data,in
the Web linked
to other data.
Both the
consumer and
the publisher
benefit from the
network effect.”
https://data.europa.eu/euodp/en/data/dataset/jrc-names
link your data to
other data to provide
context

Why are Open Data useful?
29

The value is in its use
30
http://clicnews.ie/tag/lego/

Open Data Benefits
● The Open data are the knowledge base to:
● Improve the economic grow and the
entrepreneurship based on the development of
digital services reusing Public Sector Information
● Answer to social needs through the publication of
innovative services and applications
● Aims at reducing the cost of the public
administrative activities within Public – Private
Partnerships (PPP)
● Improve the transparency of the activities of the
public institutions and the participation of the
citizens to these activities
31

Economic Growth
“Today, the cumulative value
of products and services
derived from open access
to weather data is
estimated at $15 billion.”

Potential value in Open Data
($billions)
33

Innovation: new visualizations
http://wheredoesmymoneygo.org/
34

How are Open Data related to Open
Government Data and Big Data?
35

Open Government Data
36
“The three principles of
transparency, participation,
and collaboration form the
cornerstone of an open
government”
Barack Obama, 8/12/2009
https://www.whitehouse.gov/sites/default/files/omb/assets/memoranda_2010/m10-06.pdf

Open Government Data
37

Big Data & Open Data
Variety
Volume Velocity
• Structured
• Unstructured
• Semi-structured
• …
• Terabytes
• Records
• Transactions
• Tables, Files
• Batch
• Real Time
• Streams
• Near-time
3V’s
38
Open Data is often one of
the sources for Big Data

Using Open Data: a practical
example
39

Open Data ARPAV: PFAS
40

PFAS table
41

I created a map (with QGIS)
42

Via Ca' Bianca (Campagnola)
43

Google streetview
44

Catullo Lab
45

To verify: nearer values are low
46

Your Open Data (data provider)
47
● Do you think you could be an Open Data
provider? E.g. with the datasets of your
thesis?
● Would you like to share them openly?
● If not, why?

Your Open Data (data consumer)
48
● Which data are you working on?
○ Where do you get them from?
● Which data would you like to find on
Internet?
○ Are the dataset you download fine with
you? If not, why?

Questions?
About Open Data
49
Next part: 2 - Building Open Data
frameworks

Part 2: Building Open Data
frameworks
● Open Data Issues
● Two experiences:
○ Autonomous Province of Trento
■ The story started with GeoData…
■ Now “Open Data in Trentino”: http://dati.trentino.it
○ European Commission: Joint Research Centre
■ http://data.jrc.ec.europa.eu
● The Open Data movement
● Want to learn more?
50

“Yeahh!!!”
https://unsplash.com/@littleppl85
51

LegalOrganizational TechnicalAdoption
Barriers
Contextual
52
http://goo.gl/9dFm9v
“Ohoh!!!”

Organizational Barriers
● Not ready
● Lack of resources (IT, Human)
● Don’t want to be ready
http://montcomediation.org/images/MCMC_MyWayYourWay.jpg

Legal barriers
● Open the Data
○ All the data that was produced using public money has to be
made publicly available (with exceptions)
● vs Privacy
○ You cannot open data that could allow correlation of private
personal data
http://s177.photobucket.com/user/sealth2828/media/gavel.jpg.html

● Data is not contextualized
● Opening data is a complex task, opening
cleaned data is even more complex.
● Unclear licenses
Adoption barriers
http://www.thepadrino.com/2011/01/defendius-labyrinth-security-lock.html

Technical Barriers
● Access to data:
○ Organizational
○ Technical, Downtimes,
logins,
○ Payment fees
● Fragmentation,
incomplete data,
scattered
● Format
● Cataloging,
indexing, search
● Lack of explicit
semantics,
metadata
● Conflicting
standards, models,
ontologies

● Privileged access to data
● Transparency is bad for fraudulent business
Context Barriers
http://img.gawkerassets.com/img/182n8vzdlg1iojpg/original.jpg

● Zuiderwijk et al 2010
● Listed 118 socio-technical impediments for
opening data in the literature such as:
○ Findability
○ Usability
○ Understandablity
○ Quality
○ Linking
○ Comparability and compatibility
○ Metadata
○ ….
Barriers

Congratulation for the presentation! I am curious about the data
you used! Are these datasets freely available? Would you like
to publish them as Open Data in the catalog we are creating at
the JRC level? Here there is a draft version:
http://data.jrc.ec.europa.eu/ .
Cheers,
Lorenzino
-------------------
Hi Lorenzino,sorry but I am not allowed to publish my dataset.
Cheers,
Xyz
Meanwhile at the JRC… The Data
are MINE !

Not Exactly...

Lorenzino Vaccari, Juan Pane 22/06/201661

How to deal with Open Data issues?

Autonomous Province of Trento
● The story started with GeoData…
● Now “Open Data in Trentino”:
http://dati.trentino.it
● Community building
63

The story started with GeoData …
http://www.territorio.provincia.tn.it

5 Stars Linked Geo Data Catalog
http://www.territorio.provincia.tn.it

The “Open Data in Trentino” project
66
• The “Open Data in Trentino” project is a 3 years initiative
finalized to develop an open data infrastructure to
enhance Service Innovation for Trentino following the
PAT strategy for services innovation enabled by ICT. The
project will be developed within a partnership between
Trento RISE and the Autonomous Province of Trento
(PAT) according to the innovation PAT model
• Goals
• Improved quality of life for citizens
• Open Data and local businesses
• Transparency
• Improved efficiency and productivity

Adopted licences
08/10/2013Juan Pane, Lorenzino Vaccari67
CC0 CCBY
Permissions: share, create, adapt Permissions: share, create, adapt
Actual interoperability! Decent interoperability
Constraints: nothing! Constraints: attribution

Catalogue
08/10/2013Juan Pane, Lorenzino Vaccari69
The Open Knowledge Foundation (OKF) is a
non-profit organisation founded in 2004 and
dedicated to promoting open data and open
content in all their forms – including government
data, publicly funded research and public domain
cultural content.
http://okfn.org

http://dati.trentino.it

A provincial federated catalogue...

… inspired from the guidelines of
Lombardy!
72
http://www.agendadigitale.regione.lombardia.it/

Services & Apps (do you remeber
the cake)?
Services & Apps
http://dati.trentino.it/related

Lorenzino Vaccari, Juan Pane 22/06/2016
Create Community
74
http://media.gettyimages.com/photos/members-of-the-colla-vella-de-valls-climb-up-as-they-construct-a-picture-id153610809

Trentino Open Data (TOD)
76
https://www.facebook.com/groups/todgroup/

Lorenzino Vaccari 14/06/201780 Lorenzino Vaccari

European Commission: Joint
Research Centre
● http://data.jrc.ec.europa.eu
81

JRC and Open Access
82
● for scientific publications/data within Horizon 2020 and by
other relevant initiatives (e.g. Research Data Alliance)
● overall trend for public move to open data (G8 charter,
INSPIRE..)
As continuation of JRC's efforts to make available
and transparent to the public the scientific
knowledge produced, in 2014 the JRC will roll out
its Open Access strategy for its publications
JRC Management Plan 2014
Commission Decision on the
reuse of Commission
documents (2011/833/EU)
Open Access in EC and beyond
, Anders Friis-Christensen, Andrea Perego

JRC Open Data project
83
JRC Data Policy
- Open Data principles
- Data acquisition
principles
- Data management
principles
- Implementation
principles
JRC Data Catalogue
Containing JRC datasets
related to, e.g., Soil,
Water, Air quality, Marine,
Biodiversity, etc.
http://data.jrc.ec.europa.eu
.
EU Open Data
portal
A single access point to a
growing range of data
from the institutions and
other bodies of the EU
https://open-data.europa.eu
Commission
Decision on the
reuse of
Commission
documents
(2011/833/EU)

Lorenzino Vaccari, Anders Friis-Christensen 22/06/201684 Lorenzino Vaccari, Anders Friis-Christensen 22/06/2016

Lorenzino Vaccari, Anders Friis-Christensen 22/06/201685
A JRC data infrastructure

Lorenzino Vaccari, Anders Friis-Christensen 22/06/201686
Project Scope
JRC Data Policy
Data Policy
Implementati
on Guidelines
Software
components
(e.g. data
dissemination)
Data
Open Data
Applies to

http://dati.jrc.ec.europa.eu

Using Open Data: the Open Data
movement
What is happening around us ? Some examples...
● Globally
● Europe
● Italy
● Locally
89

Open Data Charter - G8 (12/07/2013)
The principles are:
● Open Data by Default
● Quality and Quantity
● Useable by All
● Releasing Data for
Improved
Governance
● Releasing Data for
Innovation
http://opensource.com/government/13/7/open-data-charter-g8
https://www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex
90

http://census.okfn.org/
OGD around the world
91

The GEOSS portal
92
The GEOSS CORE data
principles
● Full and Open Exchange of
Data, recognizing Relevant
International Instruments
and National Policies
● Data and Products at
Minimum Time delay
● Free of Charge or minimal
Cost for Research and
Education
http://www.geoportal.org/web/guest/geo_home

OpenStreetMap: OD & Crowdsourcing
93
OpenStreetMap is a free map of
the world, created by someone
like you
“OpenStreetMap project creates and
provides geographical data, such as road
maps, freely available to anyone.
Behind the establishment and growth of
the project have been restrictions on use
or availability of map information
across much of the world and the advent
of inexpensive portable satellite
navigation devices”
https://www.openstreetmap.org

An example: Lecco
94
http://tools.geofabrik.de/

OGD in Europe - Pan European
http://www.europeandataportal.eu/en/
95
Connecting Europe Facility
launches second call
(16/05/2016)
The Connecting Europe
Facility (CEF) in Telecom is an
EU programme to facilitate
cross-border interaction
between public
administrations, businesses and
citizens, through the
deployment of Digital Service
Infrastructures. One of its aims
is to support projects which
contribute to the European
ecosystem of the deployed
interoperable and
interconnected digital services.
…

OGD in Europe - EU ODP
● screenshots
http://open-data.europa.eu/
96

The INSPIRE geoportal
97
http://inspire-geoportal.ec.europa.eu/discovery/

OGD in Italy
http://www.dati.gov.it
98

OGD in Lombardy
99
https://www.dati.lombardia.it/

Open Data @Lecco?
100
● Search if Lecco has an official Open Data
web site:
○ Which datasets (domains)?
○ Which formats?
■ How many stars?
○ Which licenses?
■ Is it clear the type of license for each dataset?
● Are there any other web sites in Lecco?
○ Are there any Universities which share
Open Data?

Want to learn more?
101

http://opendatahandbook.org/pt_BR/

The EU ODP Training
103
http://www.europeandataportal.eu/elearning/

http://schoolofdata.org/

http://www.theodi.org/

http://www.socrata.com/open-data-field-guide/

Open Data event in Lecco (8/6/2016)
107
http://www.comune.lecco.it/index.php/archivio-news/23-news-dal-comune/2437-convegno-open-data-e-sharing-economy

Open Data & Smart Cities (EU ODP)
108
Analytical Report 4: Open Data in Cities
http://www.europeandataportal.eu/sites/default/files/edp_analytical_report_n4_-_open_data_in_cities_v1.0_final.pdf

Lorenzino Vaccari 14/06/2017109 https://youtu.be/o-OyGDCbu88

Questions?
110
Thanks For Your Attention!!!!
Thanks to:
● Anders Friis-Christensen
● Maurizio Napolitano
● Juan Pane
● Andrea Perego
Lorenzino Vaccari
lorenzino.vaccari@gmail.com
lorenzino.vaccari@ec.europa.eu

About opendata2017

More Related Content

Similar to About opendata2017

More from European Commission, Joint Research Centre

Recently uploaded

About opendata2017