This document discusses open, connected, and smart cultural heritage. It proposes using crowdsourcing and machine-human computation to support multiple perspectives in interpreting digital cultural content. Disagreement between human annotators is seen as essential for helping machines with semantic interpretation. The CrowdTruth platform and software is presented as a way to harness perspectives through crowdsourced annotation and analytics. Specific use cases are described for exploring audiovisual archives about historical events in the Netherlands.

1. Open, Connected & Smart Heritage:
Towards New Cultural Commons
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

2. massive
amount
of
digital
content
to
explore
…
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

3. at
some
point
it
all
looks
the
same
…
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

4. audiences
feel
disconnected
&
lost
…
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

5. SMART
We
need
more
of
this
Johan
Oomen,
Lora
Aroyo
(2011).
Crowdsourcing
in
the
Cultural
Heritage
Domain:
OpportuniCes
and
Challenges,
hDp://www.iisi.de/fileadmin/IISI/upload/2011/p138_oomen.pdf
CONNECTED
OPEN

6. Digital
Humani;es
perform
interpreta(on
of
texts
and
other
media
interpretaCons
deal
with
perspecCves
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

7.
we
need
….
support
of
mul6ple
perspec6ves
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

8. CrowdTruth.org
Machine-­‐Human
Computa;on
for
Harnessing
Perspec;ves
in
Seman;c
Interpreta;on
Oana
Inel,
Khalid
Khamkham,
Ta6ana
Cristea,
Anca
Dumitrache,
Arne
Rutjes
,
Jelle
v.d
Ploeg
,
Lukasz
Romaszko,
Lora
Aroyo,
Robert-­‐Jan
Sips

9. HUMAN DISAGREEMENT IS
ESSENTIAL IN HELPING MACHINES
WITH SEMANTIC INTERPRETATION!
Lora
Aroyo,
Chris
Welty:
Truth
is
a
Lie:
7
Myths
about
Human
Annota6on,
AI
Magazine
2015.
(In
press)

10. • Machine Pre-processing:
op6mizing
crowdsourcing
• Micro-task Templates:
reuse
&
op6miza6on
• CrowdTruth Analytics:
disagreement-­‐based
metrics
CrowdTruth
SoFware
Components:
Machines
&
Crowds
Workflow
Ø Novel
approach
to
ground
truth
data
collec6on
&
evalua6on
Ø PROV
for
tracking
versions
of
data
Ø Reusability
in
variety
of
annota;on
tasks
&
domains
with
text,
image,
video
(thinking
about
sound)
Oana
Inel,
Khalid
Khamkham,
Ta6ana
Cristea,
Arne
Rutjes,
Jelle
van
der
Ploeg,
Lora
Aroyo,
Robert-­‐Jan
Sips,
Anca
Dumitrache
and
Lukasz
Romaszko:
Crowd
Truth:
Machine-­‐Human
Computa6on
Framework
for
Harnessing
Disagreement
in
Gathering
Annotated
Data.
ISWC-­‐RBDS

11. CrowdTruth
SoFware:
Crowdsourcing
Analy;cs
Lora
Aroyo,
Chris
Welty:
The
Three
Sides
of
CrowdTruth.
J.
Human
Computa6on.
1(1).
2014.

12. • Open
source:
h]ps://github.com/CrowdTruth
• Web
service:
h]p://stable.crowdtruth.org
CrowdTruth
SoFware:
Crowdsourcing
Analy;cs

13. dive.beeldengeluid.nl
In
Digital
Hermeneu;cs
Use
Case:
Event-­‐centric
Explora6on
Sound
&
Vision
and
Royal
Library

14. dive.beeldengeluid.nl
3rd
Price
at
the
Seman6c
Web
Challenge
2014

15. OPENIMAGES.EU
• 3000
videos
• NL
Ins6tute
for
Sound
&
Vision
• mostly
news
broadcasts
DELPHER.NL
• 1.5
Million
Scans
of
• Radio
bulle6ns
• (hand
annotated)
• 1937
–
1984

16. ENTITY EXTRACTION
CROWDTRUTH.ORG
ENTITY EXTRACTION
EVENTS CROWDSOURCING AND LINKING
TO CONCEPTS THROUGH
CROWDTRUTH.ORG
SEGMENTATION & KEYFRAMES
LINKING EVENTS AND
CONCEPTS TO
KEYFRAMES

17. SIMPLE
EVENT
MODEL
(SEM),
OPENANNOTATION
(OA)
AND
SKOS
DIVE:MEDIA
OBJECT
SEM:EVEN
T
SEM:PLACE
SEM:TIME
SEM:ACTOR
SKOS:CONCEPT
OA:ANNOTATIO
N
LINKS
TO
EUROPEANA
(MULTILINGUAL)
LINKS
TO
DBPEDIA

18. Erp,
M.
van;
Oomen,
J.;
Segers,
R.;
Akker,
C.
van
de;
Aroyo,
L.;
Jacobs,
G.;
Legêne,
S;
Meij,
L.
van
der;O
ssenbruggen,
J.R.
van;
Schreiber,
G.
Automa6c
Heritage
Metadata
Enrichment
with
Historic
Events
Museums
and
the
Web
2011
h]p://www.museumsandtheweb.com/mw2011/
papers/automa6c_heritage_metadata_enrichment_with_hi

19. CLIOPATRIA TRIPLE STORE
130K
TRIPLES
(FOR
NOW)
SPARQL
ENDPOINT
HTTP://ECULTURE.CS.VU.NL:8877/DIVE/HOME

20. SPARQL
SPARQL
SPARQL
SPARQL

21. engaging
users
through
event
narratives

22. EXPLORATIVE SEARCH
• Erp, M. van; Oomen, J.; Segers, R.; Akker, C. van de; Aroyo, L.; Jacobs, G.; Legêne, S; Meij, L. van der;O
ssenbruggen, J.R. van; Schreiber, G. Automatic Heritage Metadata Enrichment with Historic Events Museums and the
Web 2011 http://www.museumsandtheweb.com/mw2011/papers/automatic_heritage_metadata_enrichment_with_hi
h]ps://www.flickr.com/photos/drainrat/14779928998/

23. DIGITAL SUBMARINE UI
h]ps://www.flickr.com/photos/benjcarson/245171885

24. INFINITY OF EXPLORATION
h]ps://www.flickr.com/photos/mibuchat/2774251415

28. DIVE+
Timeline

29. The
Team:
Victor
de
Boer,
Oana
Inel,
Lora
Aroyo,
Johan
Oomen,
Jaap
Blom,
Werner
Helmich
&
Dennis
De
Beurs,
Chiel
van
den
Akker,
Susan
Legêne,
Huub
Wijfies,
Carlos
Mar6nez
Or6z
dive.beeldengeluid.nl

30. THANK YOU!
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo