3. http://OpenDataNode.org
Introduction references
●
COMSODE: http://www.comsode.eu/
●
Open Data Node (ODN) home page: http://opendatanode.org/
●
Documentation: https://utopia.sk/wiki/display/ODN/
●
Main GitHub project: https://github.com/OpenDataNode/open-data-node
●
On-line demo: http://demo.comsode.eu/
●
Basic non-technical introduction blog post:
http://www.comsode.eu/index.php/2015/05/open-data-node-1-0-released/
●
Basic non-technical presentation:
http://www.slideshare.net/comsode/201504-odnplatformandmethodology
4. http://OpenDataNode.org
Basic functions
According to methodology intended
(mainly) for publishers of Open Data:
●
publication plan
●
preparation of publication
●
realization of publication
●
archiving
reference: http://opendatanode.org/product/methodology-for-od-publishing/
6. http://OpenDataNode.org
most common ETL use-cases: 2* -> 3*+
(i.e. getting from non-open to Open)
●
input: XLS, SQL DB, ...
●
transformations: XLS, SQL -> CSV, „bad CSV“ -> CSV, CSV -> Linked Data
●
output:
– tabular/relational data: CSV, REST API
– Linked Data: RDF, SPARQL endpoint
Open Data
not
Open Data
Basic functions
7. http://OpenDataNode.org
Deployment strategies
ODN can be used by:
●
data publishers
●
data users
Many publishers are also users, thus
the data ecosystem is quite
complex.
ODN can be used in many roles
within that ecosystem.
more details: http://opendatanode.org/wp-content/uploads/201505-ODN_deployment_in_pilots.pdf
17. http://OpenDataNode.org
Technologies used
●
UnifiedViews:
extraction,
transformation and
enrichment of internal data
●
PostgreSQL, Virtuoso, Sesame: storage
of resulting Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint: management functions
●
CAS: SSO (internal part)
●
main component: UnifiedViews
– http://unifiedviews.eu/
●
license: combination of GPLv2 and LGPLv3
●
developed in: Java
●
other technologies: Vaadin, OSGI, ...
18. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso,
Sesame: storage of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint: management functions
●
CAS: SSO (internal part)
●
main component: PostgreSQL
– http://www.postgresql.org/
●
license: MIT/BSD style
●
developed in: C
19. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso,
Sesame: storage of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint: management functions
●
CAS: SSO (internal part)
●
main component: Virtuoso Open Source
– http://virtuoso.openlinksw.com/dataspace/doc/dav/wiki/Main
●
license: GPLv2
●
developed in: C
20. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso,
Sesame: storage of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint: management functions
●
CAS: SSO (internal part)
●
main component: Sesame (OpenRDF)
– http://rdf4j.org/
●
license: BSD style
●
developed in: Java
21. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso, Sesame: storage
of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging
functionality
●
midPoint: management functions
●
CAS: SSO (internal part)
●
main component: CKAN
– http://ckan.org/
●
license: AGPLv3
●
developed in: Python
22. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso, Sesame: storage
of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint:
management functions
●
CAS: SSO (internal part)
●
main component: modPoint
– https://evolveum.com/midpoint/
●
license: APLv2
●
developed in: Java
23. http://OpenDataNode.org
Technologies used
●
UnifiedViews: extraction, transformation
and enrichment of internal data
●
PostgreSQL, Virtuoso, Sesame: storage
of resulting
Open Data
●
CKAN, Vistuoso: publishing of stored
Open Data on the Web
●
CKAN: cataloging functionality
●
midPoint:
management functions
●
CAS: SSO (internal part)
●
main component: CAS
– https://www.apereo.org/projects/cas
●
license: APLv2
●
developed in: Java
26. http://OpenDataNode.org
Integration with Open Data Node
data harvesting side: as implied by most common use-cases
●
files: XLS, „bad CSV“, ... - almost anything(*)
●
API: SQL, SOAP, ... - almost anything(*)
●
plus all the „Open Data files and APIs“
(*) given a prominence of a format/technology or particular interest of „customer“
27. http://OpenDataNode.org
Integration with Open Data Node
special cases:
●
ODN/Management: integration of SSO with your existing infrastructure
●
ODN/Storage: direct access to SPARQL endpoint or SQL database
●
ODN/InternalCatalog: direct access to management API
●
etc.
28. http://OpenDataNode.org
Open Source
Key point, giving advantages:
●
easier to customize
●
re-use of existing tools, avoiding reinvention of the wheel
●
lower chance of vendor lock-in
●
more transparent (advantage also in public procurements)
●
etc.
29. http://OpenDataNode.org
Example of usage
in eDemokracia project, ODN is used as:
●
centralized component
●
de-centralized component
de-centralized component
centralized component
30. http://OpenDataNode.org
Example of usage
ODN as part of centralized component:
●
heavily customized
– only some modules used, commercial version of triplestore,
clustered RDBMS, etc.
●
decomposed to multiple servers
●
integrated with other components
– centralized SSO, OCR and content clasification services, etc.
●
an “upgrade” for existing data portal
data.gov.sk
– nation wide Open Data infrastrucutre
●
incorporated as extension into top-level GOV portal
slovensko.sk
31. http://OpenDataNode.org
Example of usage
ODN as de-centralized component:
●
ODN with little customizations
– central catalog and storage preconfigured
– etc.
●
distributed as „live DVD“
●
for gov. organizations and
municipalities