1.
Turning
agro-­‐biodiversity
open
data
to
real
services:
an
intro
to
technologies
Giannis
Stoitsis
Agro-­‐Know
Technologies

2.
AIM
OF
THE
COURSE

3.
How
to
get
as
much
as
possible
from
data?
Which
technologies
and
how
they
can
be
used?

4.
Course
syllabus
• Intro
• Technologies
• Sharing
data
• Linking
data
• Architectural
aspects
• Building
a
simple
case
• Some
suggesFons

5.
ARE
WE
READY
TO
BUILD
REAL
SERVICES
USING
AGRO
AND
BIODIVERSITY
OPEN
DATA?

6.
What
we
need
data
technology
and
of
course
a
good
idea

7.
As
for
the
data
Thanks
Nikos
“Intro
to
green
data
ecosystems:
why
is
a
data-­‐
powered
tech
start
up
for
agriculture
&
biodiversity
going
to
be
profitable?”

8.
TECHNOLOGIES:
THERE
IS
REALLY
GREAT
STAFF
OUT
THERE

9.
Infrastructure
• We
have
the
cloud
• Cloud
is
geJng
cheaper,
beLer
and
more
development
friendly
• OpFons
– Amazon
– Rackspace
– Linode
– MicrosoQ
– Google
– Okeanos
– Other
Private
or
even
yours
(e.g.
OpenStack,
OpenNebula)

10.
Storage and Processing Monitoring/Management/Allocation layer
Virtualization of Infrastructure Layer
Virtual Machines
Virtualization of Infrastructure LayerVirtualized Infrastractures Management Layer
GUI tools and APIs
Cloud provider A Cloud provider B Cloud provider B

11.
What
does
this
mean?
Quick
prototyping
Get
your
running
instance
in
minutes

12.
REST
• AbstracFng
the
architectural
elements
• Close
to
the
way
we
are
working
in
the
web
• Standard
hLp
methods
– GET
– PUT
– POST
– DELETE
• easy
to
be
built
especially
with
MVC
frameworks
e.g.
Rails
• easy
to
be
consumed
with
any
language
GET
hLp://www.example.com/User/1234
POST
hLp://www.example
.com/User/
‘{“user”:”{“”id”:”55”,
“name”:”John”}’
DELETE
hLp://www.example.com/User/1234

13.
REST
clients
• easy
to
be
developed
with
any
language
• check
before
start
developing

14.
Data
exchange
formats
• XML,
RDF
or
JSON?
• They
all
work
but
we
always
have
preferences
• Why
we
love
JSON?
– can
be
parsed
by
any
front
end
(web,
mobile)
– simple
– can
be
used
to
represent
data
models
– beLer
in
processing
e.g.
transformaFon
– used
widely
by
all
document
based
repositories
(e.g.
MongoDB,
elasFcsearch)

15.
Sensors
• Are
everywhere
• Two
categories
– The
one
that
we
hold
i.e.
from
smartphones
today
from
watch
tomorrow
– The
one
installed
at
sites
e.g.
fields
• We
can
get
and
use
data
from
both

16.
get
data
from
a
smartphone
• sensing
data
– MoFon
sensor/accelerometer
– Proximity
– gyroscope
(for
gaming)
• GPS:
Core
LocaFon
Framework
to
get
the
coordinates
and
tracking
user’s
locaFon

17.
GeJng
real
Fme
weather
data

18.
GeJng
real
Fme
weather
data
h:p://api.openweathermap.org/data/2.5/weather?q=London,uk

19.
Big
data
technologies
• A
trend?
• Some
of
them
– NoSQL
dbs
e.g.
MongoDB,
CouchDB
– elasFcsearch
– MapReduce
framework
e.g.
run
processes
over
a
big
amount
of
data
– Hadoop
– Cassandra
– and
more
….

20.
Natural
Language
Processing
• Dbpedia
Spotlight
• EnFty
recogniFon
(idenFfy
names
and
places
in
a
text)
• DisambiguaFon
(apple
can
be
a
company,
a
fruit
or
a
river)

21.
Image
processing
APIs
for
mobile
apps
Open
Source
Libraries

22.
how
image
processing
can
be
used?
• Similar
to
how
Google
Goggles
works
but
trained
for
your
data

23.
Machine
Learning

24.
how
machine
learning
can
be
used?
Provide
a
recommendaFon
service
Mine
and
Cluster
Data

25.
JavaScript
• can
do
almost
everything
• even
image
processing
e.g.
PixasFc
for
image
processing
• powerful
libraries
e.g.
D3
for
real
Fme
data
visualizaFon

26.
AnalyFcs
• Track
the
users
and
events
of
your
applicaFon
– ElasFc
Search,
MongoDB
– Google
AnalyFcs

27.
Follow
a
well
accepted
framework
• Model
View
Controller
(MVC)
• There
are
many
nice
MVC
based
frameworks
– Rails
– Spring
– Zend
– Yii
– Laravel
– and
more
…

28.
SHARING
DATA

29.
Publisher
Date Catalog
Subject
ID
Author
Title
we actually share metadata

30.
the
value
is
sFll
in
metadata
• Metadata
connect
us
to
the
real
data
– Search
metadata
records
not
raw
data
• Metadata
gives
the
context
– themaFc
classificaFon
– target
users
• Well
defined
standards
(e.g.
DC,
IEEE
LOM,
Darwin
Core,
ABCD)
• RDF,
XML
and
JSON
binding

31.
AGGREGATING
METADATA

32.
…
the
power
is
at
the
back
end
CulQvaQon
HarvesQng
Blossom
Unorganized
content
in
local
and
remote
sites
Organized
and
structured
content
in
local
and
remote
databases
EducaQonal
Bibliographic
EducaQonal
Geographical
Bibliographic
Agricultural
Data
PlaVorm
Aggregate
data
from
diverse
sources
Work
with
different
type
of
data
Prepare
data
for
meaningful
services
IngesQon
Enrichment
TranslaQon
Publishing
Data
discovery
services
Widgets
Authoring
services
AnalyQcs
services

33.
Why
to
aggregate?
• Enrich
metadata
• Provide
developer
friendly
APIs
that
can
be
used
to
build
good
services
• Data
providers
rarely
have
APIs

34.
LINKING
DATA

35.
Why
we
sFll
have
data
silos?
• CompeFng
metadata
standards
(e.g.
DC,
IEEE
LOM)
• Diversity
of
web
interfaces
(e.g.
REST,
OAI-­‐PMH,
SOAP,
SPI,
SQI)
• Different
exchange
format
(e.g.
XML,
RDF,
JSON)
• Fragmented
use
of
texonomies
LD for educational data/resource sharing
Overview
Approaches for LD in educational data sharing
 On the-fly/automated integration of heterogeneous APIs and data (http://www.meducator.net)
 Dataset (transformation and) cataloging (http://linkedup-project.eu)
?
We
are
sFll
here
…
…
and
not
here
…

36.
What
is
Linked
Data?
• A
set
of
principles
and
technologies
for
the
Web
of
Data
• Principles
– Put
the
data
online
with
permanent
address
(URIs)
– Describe
the
data
with
a
standard
representaFon
(RDF)
– Link
to
other
data
through
published
taxonomies
• Technologies
– Triple
stores
– SPARQL
– RDF,
OWL
– SWRL

37.
Seman&c
Web
/
Linked
Data
Technologies
by
Mathieu
d'Aquin
on
Sep
11,
2013
ApplicaFons

38.
Mash
up
and
Linked
Data:
FAO
case

39.
How
it
works?
• Based
on
AGROVOC
• Using
AGROVOC
mappings
to
other
ontologies
e.g.
GeopoliFcal
and
DBpedia
<hLp://aims.fao.org/aos/agrovoc/c_690>
<hLp://www.w3.org/2004/02/skos/core#closeMatch>
<hLp://dbpedia.org/resource/Salmon>
<hLp://dbpedia.org/page/Salmon>
replace
resource
with
page
and
get
rdf

40.
Why
to
rely
on
linked
data
• Easy
to
be
processed
e.g.
microdata,
rdf
• Get
more
out
of
data
through
linking
e.g.
get
the
staFsFcs
for
a
country
through
GeoPoliFcal
Ontology
Web
resource
about
Greece
annotated
with
Agrovoc
term
“Greece”
hLp://www.fao.org/countryprofiles/geoinfo/geopoliFcal/data/Greece
Country
GDP

41.
and
even
more
…
• Data
that
you
can
get
from
World
Bank
based
on
ISO3
country
code
– Agricultural
irrigated
land
(%
of
total
agricultural
land)
– Cereal
yield
(kg
per
hectare)
– Rural
populaFon
(%
of
total
populaFon)
– Poverty
headcount
raFo
at
rural
poverty
line
(%
of
rural
populaFon)
– Agricultural
machinery,
tractors
per
100
sq.
km
of
arable
land
– Arable
land
(%
of
land
area)
– CO2
emissions
(kt)
– Fish
species,
threatened

42.
DECOUPLING
FRONT
END
FROM
BACK
END

43.
how
it
works
Template
customizaQon
html,
css,
Ajax,
JS
Data
mash
up
Search
API
Cloud
Edu
resources
Data
processing
and
enrichment
e.g.
semanQc
enrichment
or
clustering
Search
API
Cultural
resources
from
Museums

44.
how
it
works
Template
customizaQon
html,
css,
Ajax,
JS
widget
in
Facebook
page
Data
mash
up
Search
API
Cloud
Edu
resources
Data
processing
and
enrichment
e.g.
semanQc
enrichment
or
clustering
Search
API
Cultural
resources
from
Museums

45.
AUTHENTICATION

46.
Use
authenFcaFon
APIs

47.
NATIVE
OR
WEB
APP?

48.
NaFve
vs
Web
app
• Depends
on
the
business
case
• Responsive
web
app
are
cross
plaworm
• Responsive
design
is
great
but
it
is
not
so
friendly
as
naFve
app
• NaFve
app
has
nice
workflow
• NaFve
app
can
use
device
sensors
• In
many
data-­‐powered
cases
you
need
both

49.
BUILDING
A
BASIC
APP
IN
HOURS

50.
THE
IDEA

51.
A
mobile
app
for
discovering
green
learning
resources
• Similar
apps
– OER
commons
– iMarine
• Similar
web
app
– hLp://www.greenlearningnetwork.org/
organicedunet/

52.
Which
data
• Learning
Resources
about
environment,
agriculture
and
biodiversity
• API
end
point
hLp://83.212.96.219:8080/
glnRepo/api/ariadne/restp?json=Χ&callback=Y

53.
Recipe
• Mobile
app
SDK
e.g.
XCODE
• Mobile
simulator
• Some
reading
• Similar
examples
e.g.
Searching
in
twiLer
• A
workflow
• JSON
parser
• No
beauFes
at
this
stage

54.
Development
is
friendly

55.
Defining
the
workflow
is
a
game

56.
ConnecFng
to
the
data
Use
NSJSONSerializaFon
Class
to
get
the
data

57.
and
finally

58.
SuggesFons
• focus
on
the
idea
and
business
value
• re-­‐use
the
workflow
of
successful
apps
• processing
gives
the
value
• do
not
create
unnecessary
procedures
e.g.
RegistraFon
page
• re-­‐use
exisFng
data
• put
emphasis
on
the
value
of
your
app
and
not
on
the
complexity
of
the
technology
• always
check
first
if
there
is
open
source
code
• design
maLers
• go
beyond
trivial
and
benefit
from
innovaFon

59.
thank
you!
stoitsis@agroknow.gr