Mantas Zimnickas presented on open data in Lithuania at OpenCon 2016. He discussed what open means according to opendefinition.org, defined open data, and summarized the status of open data initiatives in Lithuania. Zimnickas' site atviriduomenys.lt aims to provide open data to users by collecting data from providers and using it in projects. The talk covered challenges of working with different data formats and proposed approaches to standardizing and automating data extraction and transformation. Zimnickas concluded by sharing code for several of his open data projects.

1. Mantas Zimnickas
atviriduomenys.lt
OpenCon 2016 Lithuania

2. What is open?
“Open means anyone can
freely access, use, modify,
and share for any purpose.”
– opendefinition.org

3. What is open data?
5stardata.info

4. Open data status in Lithuania
http://ivpk.lrv.lt/lt/ivpk-leidiniai/atliktas-galimybiu-tyrimas-skirtas-apibrezti-atviru-
duomenu-iniciatyvos-igyvendinimo-lietuvoje-architekturos-modeli

5. What atviriduomenys.lt is about?
Data
providers
Data Projects
provides are used by
needsimportance
Users
likes
are used by
atviriduomenys.lt

6. Data packages
CSV/JSON RDF/DCAT
'2016-11-12' "2016-11-12"^^xsd:date
Datapackage
date(2016, 11, 12)
frictionlessdata.io/data-packages/

7. HTML data to
download first page
extract links
download all links
extract data
transform data
with index.download('http://opendata.gov.lt/'):
with urls.select('table td[3] a@href'):
with pages.download():
with raw_data.select(row.key, [
'table xpath:tr[count(td)=2]',
('td[1]:content', 'td[2]:content'),
]):
data.call(
lambda x: (x.key, dict(x.value))
)
data.export('opendatgovlt.csv')export data
atviriduomenys.lt/data

8. Data to
Raw
values
Uniform
Resource
Identifier
LRS
ld:orgs/gov/42
Canonical
object
registry
Lietuvos
Respublikos
Seimas
Seimas
LR Seimas
Aliases and
templates
Lietuvos Respublikos {name}
LR{name|abbr}
{name}
LR {name}
https://github.com/sirex/databot-bots/tree/master/index

9. Technology moves forward
people don’t
225 years
later

10. A4 paper data format
Writes on
A4 paper
Reads
A4 paper

11. In a perfect world...
Creates
ontologies
Creates
templates
Fills in
templates
with data
HTML
representation
Printing
on paper
Smartphone
app
Single
source
publishing
W
YSIW
YM
Context

12. Real world examples
Abby
eFormFiller
Akoma Ntoso
(LegalDocML)
LegisPro
By Xcential
Word processor
templates with
fields

13. Text data to
Text
stream
Pattern
recognition
engine
Templates Data
Field: value N-gram magic Field: {value} {'Field': 'value'}
automatization

14. Text data to
Text
samples
Training Data
big trees machine learning forest
NLP
a tree

15. Manuscript to
epaveldas.lt

16. Thank you!
● https://github.com/sirex/atviriduomenys.lt
● https://github.com/sirex/databot/
● https://github.com/sirex/databot-bots
● https://github.com/sirex/gramtool
● https://bitbucket.org/sirex/morfologija
● https://github.com/frictionlessdata/
Source code of atviriduomenys.lt and related projects: