When Drupal meets Opendata
PHPDay 2011 - Social tag #phpday

Chi siamo?

Chi siamo?
Stefano Mainardi,
sviluppatore PHP, CEO di
Twinbit e fondatore del
network italiano ILDN
Piacere di conoscervi!
stefano@twinbit.it
twitter : @stefanomainardi

Chi siamo?
Paolo Mainardi, sviluppatore
PHP, CTO di Twinbit e Stefano Mainardi,
fondatore del network italiano sviluppatore PHP, CEO di
ILDN Twinbit e fondatore del
network italiano ILDN
Piacere di conoscervi!
Piacere di conoscervi!
paolo@twinbit.it
stefano@twinbit.it
twitter : @paolomainardi
twitter : @stefanomainardi

Chi siamo?
Paolo Mainardi, sviluppatore
PHP, CTO di Twinbit e Stefano Mainardi,
fondatore del network italiano sviluppatore PHP, CEO di
ILDN Twinbit e fondatore del
network italiano ILDN
Piacere di conoscervi!
Piacere di conoscervi!
paolo@twinbit.it
stefano@twinbit.it
twitter : paolomainardi
twitter : stefanomainardi
Piacere, io sono Drupal! :-)

Quanti di voi utilizzano Drupal?

Cosa sono gli “Open Data”?

definizione “accademica”
Con Open Data si fa riferimento ad una filosofia, che è al tempo
stesso una pratica, che implica che alcune tipologie di dati siano
liberamente accessibili a tutti sul Web, senza restrizioni di copyright,
brevetti o altre forme di controllo che ne limitino la riproduzione.
da Wikipedia

e quindi?

http://www.flickr.com/photos/f7oor/405046410/

Data is a precious thing and will last longer than the systems
themselves.
Tim Berners-Lee
http://www.flickr.com/photos/f7oor/405046410/ 1/2

I dati sono noiose scatole marroni, ed è così che ce li
immaginiamo, no? Perché i dati di per sé non sono di
immediata applicazione Ma in realtà, i dati determinano
tantissime cose nelle nostre vite e ciò accade perché c’è
qualcuno che prende quei dati e ne fa qualcosa.
credits: Ernesto Bellisario “Come si fa OpenData”
Tim Berners-Lee
http://www.flickr.com/photos/f7oor/405046410/ 2/2

“Economia della conoscenza”

ma in italia cosa succede?

il movimento OpenData italiano
http://linkedopendata.it/
http://spaghettiopendata.org/
http://datagov.it/
http://www.agoradigitale.org/
http://www.openpolis.it/

quali formati dobbiamo utilizzare?

.KML .XML .CSV .TXT .GTFS .JSON

.KML .XML .CSV .TXT .GTFS .JSON
insomma, formati standard che siano machine readable
l’importante è non far nascere un flame sui formati :-)

.KML .XML .CSV .TXT .GTFS .JSON

.KML .XML .CSV .TXT .GTFS .JSON
Mobile Apps

.KML .XML .CSV .TXT .GTFS .JSON
Mobile Apps Web Apps

.KML .XML .CSV .TXT .GTFS .JSON
Mobile Apps Web Apps Data API

I dati sono il carburante, le applicazioni sono il mezzo

47 web, iPhone & Facebook Apps
$2,300,000 risparmiati
$50,000 il costo delle applicazioni
+5000% in ROI

Perchè Drupal ?

Perchè Drupal ?
• Una piattaforma pensata per fare Open Data (D7 RDFa, Nodes, Taxonomy)

Perchè Drupal ?
• Una piattaforma pensata per fare Open Data (D7 RDFa, Nodes, Taxonomy)
• Semplici interfacce di amministrazione per utenti non esperti (D7 UX)

Perchè Drupal ?
• Una piattaforma pensata per fare Open Data (D7 RDFa, Nodes, Taxonomy)
• Semplici interfacce di amministrazione per utenti non esperti (D7 UX)
• Semplice integrazione con i servizi di terze parti (CKAN, OpenCalais, OSM, Mapbox ecc..)

Perchè Drupal ?
• Una piattaforma pensata per fare Open Data (D7 RDFa, Nodes, Taxonomy)
• Semplici interfacce di amministrazione per utenti non esperti (D7 UX)
• Semplice integrazione con i servizi di terze parti (CKAN, OpenCalais, OSM, Mapbox ecc..)
• Importanti case history governative di successo (List of Drupal sites in
Government: http://groups.drupal.org/node/19885)

Perchè Drupal ?

Perchè Drupal ?
•CCK - Fields in core

Perchè Drupal ?
•CCK - Fields in core
•Feeds

Perchè Drupal ?
•CCK - Fields in core
•Feeds
•Views - Views datasource

Perchè Drupal ?
•CCK - Fields in core
•Feeds
•Views - Views datasource
•Geo Modules (Openlayers, Geofield)

Perchè Drupal ?
•CCK - Fields in core
•Feeds
•Views - Views datasource
•Geo Modules (Openlayers, Geofield)
•Supporto nativo per i Linked Data (RDFa)

Perchè Drupal ?
•CCK - Fields in core
•Feeds
•Views - Views datasource
•Geo Modules (Openlayers, Geofield)
•Supporto nativo per i Linked Data (RDFa)
•SPARQL

Case history di successo
http://data.worldbank.org

Case history di successo
http://www.aquaknow.net

Case history di successo
http://data.gov.uk
CKAN Dataset
Faceted Search by Apache Solr

Case history di successo
http://www.openbelgium.be

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)
Seguiamo queste semplici regole:

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)
Seguiamo queste semplici regole:
•Specificare la fonte

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)
Seguiamo queste semplici regole:
•Specificare la fonte
•Usare una licenza aperta

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)
Seguiamo queste semplici regole:
•Specificare la fonte
•Usare una licenza aperta
•Usare formati aperti e semplici da processare automaticamente (CSV, XLS,
XML, JSON)

Phase 1: Creare Open Data
Qualsiasi contenuto è potenzialmente un OpenData.....
“A piece of content or data is open if anyone is free to use, reuse, and redistribute it — subject only, at most, to the
requirement to attribute and share alike.” (Open Knowledge Definition)
Seguiamo queste semplici regole:
•Specificare la fonte
•Usare una licenza aperta
•Usare formati aperti e semplici da processare automaticamente (CSV, XLS,
XML, JSON)
OpenDefinition: http://www.opendefinition.org/okd/
Open Data Commons: http://www.opendatacommons.org
Cretive Commons: http://www.opendatacommons.org

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
DataTypes Vocabulary
License Vocabulary

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
DataTypes Vocabulary
License Vocabulary
Fields

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
Open data creation form

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
Open data creation form

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
Open data creation form
Taxonomy

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
Open data creation form
Taxonomy
Data properties

Phase 1: Creare Open Data
Creare un nuovo content type ed un vocabolario
Open data creation form
Taxonomy
Data properties
Comments

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core
x/html

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core
x/html
RDFa

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core
x/html RSS
RDFa

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core contrib modules
x/html RSS json
xml
rdf
RDFa csv

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core contrib modules
x/html RSS json
xml
rdf
RDFa csv
Views + Views Datasource

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core contrib modules
charts
x/html RSS json kml {$other formats}
xml
rdf
RDFa csv
Views + Views Datasource

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
core contrib modules
charts
x/html RSS json kml {$other formats}
xml
rdf
RDFa csv http://drupal.org/project/modules
Views + Views Datasource

Phase 2: Distribuire Open Data
Come posso distribuire i miei dati ?
Posso essere anche solo un servizio per la tua applicazione:
http://drupal.org/project/services
XMLRPC, JSON, JSON-RPC, REST, SOAP, AMF, etc.
core contrib modules
charts
x/html RSS json kml {$other formats}
xml
rdf
RDFa csv http://drupal.org/project/modules
Views + Views Datasource

Phase 2: Distribuire Open Data
Views Example: Create a new display

Phase 2: Distribuire Open Data
Views Example: Create a new display
Filter by “data” type contents

Phase 2: Distribuire Open Data
Views Example: Create a new display
Select fields to print
Filter by “data” type contents

Phase 2: Distribuire Open Data
Views Example: Create a new display
Print results on a table
Select fields to print
Filter by “data” type contents

Phase 2: Distribuire Open Data
Views Example: Create a new display
Print results on a table
Select fields to print
Filter by “data” type contents
Expose search fields

Phase 2: Distribuire Open Data
Views Example: Create a new display
Output type
Print results on a table
Select fields to print
Filter by “data” type contents
Expose search fields

Phase 2: Distribuire Open Data
Views Example: Results

Phase 2: Distribuire Open Data
Views Example: Results
Results

Phase 2: Distribuire Open Data
Views Example: Results
Exposed form
Results

Phase 2: Distribuire Open Data
Views Example: Results
Exposed form
Results
RSS Export

Phase 2: Distribuire Open Data
Views Example: Results
Exposed form
Results
RSS Export Click to download in other formats!

Phase 2: Distribuire Open Data
Example: Geo tools

Phase 2: Distribuire Open Data
Openlayers module
Example: Geo tools

Phase 2: Distribuire Open Data
Openlayers module
Example: Geo tools
Drupal nodes

Phase 2: Distribuire Open Data
Example: Geo tools
Layer manager

Phase 2: Distribuire Open Data
Example: Geo tools
Layer manager

Phase 2: Distribuire Open Data
Example: Geo tools Views Datasource (json)
Layer manager

Phase 2: Distribuire Open Data
Example: Apache Solr -- http://drupal.org/project/apachesolr

Phase 2: Distribuire Open Data
Used on Drupal.org Example: Apache Solr -- http://drupal.org/project/apachesolr

Phase 2: Distribuire Open Data
Used on Drupal.org Example: Apache Solr -- http://drupal.org/project/apachesolr
Faceted Search

Phase 2: Distribuire Open Data
Used on Drupal.org Example: Apache Solr -- http://drupal.org/project/apachesolr
• Powerful text extraction Faceted Search
• Multisite search
• Attachment search

Phase 3: Data processing
Feeds: http://drupal.org/project/feeds

Phase 3: Data processing
Feeds: http://drupal.org/project/feeds
Pluggable fetcher

Phase 3: Data processing
Feeds: http://drupal.org/project/feeds
Pluggable fetcher
Pluggable parser

Phase 3: Data processing
Feeds: http://drupal.org/project/feeds
Pluggable fetcher
Pluggable parser
Pluggable processors

Phase 3: Data processing
Feeds: http://drupal.org/project/feeds
Pluggable fetcher
Pluggable parser
Pluggable processors

Phase 3: Data processing
Feeds Parser
Feeds Xpath Parser: http://drupal.org/project/feeds_xpathparser
Feeds QueryPath Parser: http://drupal.org/project/feeds_querypath_parser
Data importing projects
http://drupal.org/project/migrate
“The migrate module provides a flexible framework for migrating content into Drupal from other sources (e.g., when converting a web site from another CMS to Drupal)”
http://drupal.org/project/data
“Data module helps you model, manage and query related sets of tables. It offers an administration interface and a low level API for manipulating tables and accessing their contents. D
module provides Views integration for displaying table data and Drupal search integration for searching table content.”

Linked Data
http://groups.drupal.org/semantic-web
http://www.w3.org/2001/sw/

RDF
http://openspring.net/blog/2011/05/01/background-research-work-leading-to-rdf-in-drupal-7-released-as-part-of-my-masters

RDF
http://drupal.org/project/rdfx
http://drupal.org/project/rdf (d6)
API http://api.drupal.org/api/drupal/modules--rdf--rdf.module/group/rdf/7

RDF
http://drupal.org/project/rdfx
http://drupal.org/project/rdf (d6)
API http://api.drupal.org/api/drupal/modules--rdf--rdf.module/group/rdf/7
• RDFx: extends core RDF support by providing extra APIs and additional serialization formats such as RDF/XML,
NTriples, Turtle...

RDF
http://drupal.org/project/rdfx
http://drupal.org/project/rdf (d6)
API http://api.drupal.org/api/drupal/modules--rdf--rdf.module/group/rdf/7
• RDFx: extends core RDF support by providing extra APIs and additional serialization formats such as RDF/XML,
NTriples, Turtle...
• RDF UI: allows site administrators to specify the RDF mappings via a user interface

RDF
http://drupal.org/project/rdfx
http://drupal.org/project/rdf (d6)
API http://api.drupal.org/api/drupal/modules--rdf--rdf.module/group/rdf/7
• RDFx: extends core RDF support by providing extra APIs and additional serialization formats such as RDF/XML,
NTriples, Turtle...
• RDF UI: allows site administrators to specify the RDF mappings via a user interface
• Evoc: user interface to import vocabularies which can be used in RDF UI

RDF
http://drupal.org/project/rdfx
http://drupal.org/project/rdf (d6)
API http://api.drupal.org/api/drupal/modules--rdf--rdf.module/group/rdf/7
• RDFx: extends core RDF support by providing extra APIs and additional serialization formats such as RDF/XML,
NTriples, Turtle...
• RDF UI: allows site administrators to specify the RDF mappings via a user interface
• Evoc: user interface to import vocabularies which can be used in RDF UI
This RDF data can be made available in a SPARQL endpoint with the SPARQL module.

SPARQL
http://drupal.org/project/sparql
SPARQL is a query language for pattern matching against RDF graphs. The syntax resembles SQL, making for a friendly learning
curve, but SPARQL is far more powerful, enabling queries spanning multiple disparate (local or remote) data sources containing
heterogeneous semistructured data.

SPARQL
http://drupal.org/project/sparql
SPARQL is a query language for pattern matching against RDF graphs. The syntax resembles SQL, making for a friendly learning
curve, but SPARQL is far more powerful, enabling queries spanning multiple disparate (local or remote) data sources containing
heterogeneous semistructured data.
•SPARQL API module

SPARQL
http://drupal.org/project/sparql
SPARQL is a query language for pattern matching against RDF graphs. The syntax resembles SQL, making for a friendly learning
curve, but SPARQL is far more powerful, enabling queries spanning multiple disparate (local or remote) data sources containing
heterogeneous semistructured data.
•SPARQL API module
•SPARQL Registry module

SPARQL
http://drupal.org/project/sparql
SPARQL is a query language for pattern matching against RDF graphs. The syntax resembles SQL, making for a friendly learning
curve, but SPARQL is far more powerful, enabling queries spanning multiple disparate (local or remote) data sources containing
heterogeneous semistructured data.
•SPARQL API module
•SPARQL Registry module
•SPARQL Endpoint module

LOD Cloud
“Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/”

References
The Semantic Web, Linked Data and Drupal, Part 1: Expose your data using RDF
http://www.ibm.com/developerworks/web/library/wa-rdf/index.html?ca=drs-
The Semantic Web, Linked Data and Drupal, Part 2: Combine linked datasets with Drupal 7 and SPARQL Views
http://www.ibm.com/developerworks/web/library/wa-rdf/index.html?ca=drs-
Background research work leading to RDF in Drupal 7 released as part of my Master's thesis
http://openspring.net/blog/2011/05/01/background-research-work-leading-to-rdf-in-drupal-7-released-as-part-of-my-masters
Background research work leading to RDF in Drupal 7 released as part of my Master's thesis
http://openspring.net/blog/2011/05/01/background-research-work-leading-to-rdf-in-drupal-7-released-as-part-of-my-masters
“Hi, I'm Lin Clark. I support open source and linked data.”
http://lin-clark.com/

one more thing
è in arrivo il primo contest italiano sugli OpenData
http://www.appsforitaly.org/

Questions?

Rimaniamo in contatto!
Stefano Mainardi Paolo Mainardi
stefano@twinbit.it paolo@twinbit.it
twitter : @stefanomainardi twitter : @paolomainardi

Grazie!