This document summarizes Lora Aroyo's presentation on enabling digital humanities. The presentation discusses expanding human cognition with technology, teaching machines through crowdsourcing diverse interpretations, and engaging users through novel interfaces like event-based browsing of linked historical media. The goals are to support interpretation, gather human semantics, and allow natural interaction. Challenges and the road ahead are also outlined.

1. Expand.
Learn.
Interact.
Enabling
(Digital)
Humani9es
Lora
Aroyo
VU
University
Amsterdam
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

2. “The
Gallery
of
Cornelis
van
der
Geest”
(Willem
van
Haecht)
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

3. is
a
rich
digital
space
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

4. to
explore
TOPICS,
e.g
hun9ng,
dogs

5. Quen9n
Metsys
The
Madonna
and
Child
to
explore
ARTISTS,
e.g.
their
pain9ngs,
biographies,
links

6. to
explore
EVENTS,
e.g
types,
par9cipants,
links

7. Mul9tude
of
collec9ons
Mul9-­‐
dimensional
Mul9-­‐
perspec9ve
“Jungle
of
scales”
Mul9-­‐granularity
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

8. Digital
Heritage
Mission
“We
enable
everyone
to
u/lize
the
collec/ons
Lora Aroyo
to
learn,
experience
&
create.”
(mission
Sound
and
Vision)
“a
place
where
a
na/on
nourishes
its
memory,
and
exerts
its
imagina/on
-­‐
where
it
connects
with
its
past
and
invents
its
future”
(Pierre
Ryckmans,
1996)
„places
that
foster
curiosity,
celebrate
ideas
and
enable
discovery.”
(sxswLAM
community,
hHelp
people
Mp://www.sxswlam.com/)
finding
their
way

9. Informa(on:
Museums
&
Archives
as
Inventories
of
the
World
André
Malraux,
The
Imaginary
Museum
of
World
Sculpture,
1953

10. Interpreta(on:
Museums
&
Archives
as
a
Place
to
Engage
with
the
World

11. effec/ve
&
efficient
in
delivering
change
and
tangible
benefits
strategic
advantage
through
the
innova/on
inherent
in
this
digital
ac/vity
economic
benefit
to
community
that
demonstrate
the
value
of
endeavours
in
monetary
terms
(
the
community
has
been
changed
by
the
resource
in
beneficial
ways
that
can
be
clearly
iden/fied
Source:
Simon
Tanner,
Kings
College
London
,
hBp://www.kdcs.kcl.ac.uk/innovaGon/impact.html
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

12. New
Era
of
Compu9ng
• amount
Lora Aroyo
of
data
&
scale
of
computa9on
available
have
increased
by
a
previously
inconceivable
amount
• CS
&
AI
moved
out
of
thought
problems
–
entered
empirical
science

14. Open Domain Question-Answering Machine, that given
– Rich Natural Language Questions
– Over a Broad Domain of Knowledge
Won a 2-game Jeopardy match against the all-time winners
– viewed by over 50,000,000
IBM
Confiden/al

15. Enabling
(Digital)
Humani9es
Expand.
Learn.
Interact.
How
to
Do
It?
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

16. 1.
Expand
“EXPAND
human
cogni/on,
makes
the
jobs
we
do
easier,
like
a
cogni9ve
Lora Aroyo
prosthesis,
especially
when
dealing
with
processing
massive
data,
or
data
that
requires
human
interpreta9on”
• Support
for
Interpreta9on
• Structured
Seman9cs

17. … Digital Humanities researchers
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

18. … often don’t find
what they were searching for
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

19. HOW
DO
WE
FILL
THE
GAP?
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

20. Concep9ons
of
eHuman9es
INTERPRETATION

21. • a
Digital
Hermeneu9cs
theory
of
interpretaGon
of
informaGon
• bringing
people
and
technology
together
to
explore:
– how
to
model
and
represent
informa/on
– how
to
provide
engaging
interac/on
– how
to
support
interpreta/on
“Digital
HermeneuGcs:
Agora
and
the
online
understanding
of
cultural
heritage”
In
proceedings
of
Web
Science
Conference,
(ACM:
New
York,
2011)

22. Simple
Event
Model
(SEM)
OpenAnnota/on
(OA)
&
SKOS
DIVE:MEDIA
OBJECT
SEM:EVEN
T
SEM:PLACE
SEM:TIME
SKOS:CONCEP
T
SEM:ACTOR
OA:ANNOTATIO
N
• Links
to
EUROPEANA
(mul/lingual)
• Links
to
DBPedia,
other
vocabularies

23. Erp,
M.
van;
Oomen,
J.;
Segers,
R.;
Akker,
C.
van
de;
Aroyo,
L.;
Jacobs,
G.;
Legêne,
S;
Meij,
L.
van
der;O
ssenbruggen,
J.R.
van;
Schreiber,
G.
Automa/c
Heritage
Metadata
Enrichment
with
Historic
Events
Museums
and
the
Web
2011
hMp://www.museumsandtheweb.com/mw2011/
papers/automa/c_heritage_metadata_enrichment_with_hi

24. engaging
users
through
event
narratives

25. Exploring
with
Structured
Seman9cs
hMp://sealinc.ops.few.vu.nl/invenit/

26. CLUSTERING
! RANKING
! TRUST
! INTEREST
hMp://sealinc.ops.few.vu.nl/invenit/

27. 2.
Learn
“LEARN
as
you
use
it
–
most
machine
errors
are
easy
for
a
human
to
detect,
and
we
can
instrument
usage
of
systems
to
beMer
understand
the
system
and
the
problem
it
solves”
• Teach
Lora Aroyo
machines
• Deal
with
diversity
of
interpreta9on

28. Teaching
Machines
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

29. Diversity,
Opinions,
Interpreta9ons
Human Disagreement IS Essential in Helping
Machines with Semantic Interpretation!
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

30. Crowdsourcing
Diversity
of
Interpreta9on
“best
collec9ve
decisions
are
result
of
disagreement,
not
consensus
or
compromise”
James
Surowiecki
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

31. CrowdTruth.org
A
novel
approach
to
gather
diversity
of
perspec9ves
&
opinions
from
the
crowd,
expand
expert
vocabularies
with
these
and
gather
new
type
of
gold
standard
for
machines
L.
Aroyo,
C.
Welty:
Crowd
Truth:
Harnessing
disagreement
in
crowdsourcing
a
rela/on
extrac/on
gold
standard.
ACM
WebSci
2013.
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

32. Crowdsourcing
for
Video
Analysis
CrowdTruth.org
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

33. Following the grandeur of Baroque, Rococo art is
often dismissed as frivolous and unserious, but
Waldemar Januszczak disagrees. […] The first
episode is about travel in the 18th century and how
it impacted greatly on some of the finest art ever
made. The world was getting smaller and took on
new influences shown in the glorious Bavarian
pilgrimage architecture, Canaletto's romantic
Venice and the blossoming of exotic designs and
tastes all over Europe.
Rococo:
Travel,
pleasure,
madness
A
boarding
school
where
boys
from
the
Dutch
East
Indies
receive
a
voca/onal
educa/on
has
been
set
up
by
the
ministry
of
Social
Affairs
near
Batavia.
Trumpeter
sounds
the
reveille;
-­‐
the
boys
get
out
of
bed;
-­‐
the
muster
is
held
in
front
of
the
building;
akerwards
the
boys
stand
in
line
with
a
food
bowl;
they
get
rice
pudding
and
eat
this
around
long
tables
in
the
open
air;
-­‐
the
boys
get
instruc/ons
and
prac/ce
a.o.
drawing
shapes,
forging,
metalworking,
filing
and
woodworking;
News
from
Indonesia:
youth
care
hdp://vista-­‐tv.eu
hdp://dive.beeldengeluid.nl

34. Crowdsourcing
for
Image
Analysis
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
CrowdTruth.org

35. Crowdsourcing
for
Text
Analysis
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo
CrowdTruth.org

36. • One
truth:
data
collec/on
efforts
assume
one
correct
interpreta/on
for
every
example
• All
examples
are
created
equal:
ground
truth
treats
all
examples
the
same
–
either
match
the
correct
result
or
not
• Detailed
guidelines
help:
if
examples
cause
disagreement
-­‐
add
instruc/ons
to
limit
interpreta/ons
• Disagreement
is
bad:
increase
quality
of
annota/on
data
by
reducing
disagreement
among
the
annotators
• One
is
enough:
most
of
the
annotated
examples
are
evaluated
by
one
person
• Experts
are
beder:
annotators
with
domain
knowledge
provide
beMer
annota/ons
• Once
done,
forever
valid:
annota/ons
are
not
updated;
new
data
not
aligned
with
previous
7
Myths
of
Human
Annota9on
myths
directly
influence
the
prac/ce
of
collec/ng
human
annotated
data;
Need
to
be
revisited
in
the
context
of
new
changing
world
&
in
the
face
of
a
new
theory
of
truth
(CrowdTruth)
Lora Aroyo

37. 3.
Interact
“INTERACT
naturally.
We
need
to
bring
machines
closer
to
their
users,
we
have
adapted
ourselves
enough
to
them,
they
should
understand
natural
Lora Aroyo
language,
spoken
or
wriMen,
be
able
to
process
images
and
videos.
These
simple
human
problems
are
extremely
complex
for
machines,
but
are
hallmarks
of
a
new
compu9ng
era.”
• Gathering
Human
Seman9cs
• Engaging
Users

38. Engage
the
Crowd
in
Cultural
Heritage

39. Engage
the
Niche
hMp://2-­‐dot-­‐rma-­‐accurator.appspot.com/
in
Cultural
Heritage

40. Engage
with
Games
in
Cultural
Heritage

41. Engage
with
Games
in
Sciences
hdp://game.crowdtruth.org/
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

42. Engage
with
Games
hdp://waisda.nl/
in
Entertainment
hdp://spotvogel.vroegevogels.vara.nl/
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

43. Engage
with
Novel
Interfaces
Event-­‐based
Browsing
Of
Linked
Historical
Media

44. dive.beeldengeluid.nl
3rd
Price
at
the
Seman9c
Web
Challenge
2014

45. OPENIMAGES.EU
• 3000
videos
• NL
Ins/tute
for
Sound
&
Vision
• mostly
news
broadcasts
DELPHER.NL
• 1.5
Million
Scans
of
• Radio
bulle/ns
• (hand
annotated)
• 1937
–
1984

46. Digital
Submarine
UI
Infinity
of
Explora9on
Linked
(Open)
Data
Events
Linking
Objects
Crowd
Bringing
Human
Perspec9ves

47. Expand.
Learn.
Interact.
Enabling
(Digital)
Humani9es
The
Road
Ahead
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

48. Technology
Challenges
• Quality
Lora Aroyo
measures
• Capturing
Trust
• Dealing
with
Provenance
• Complexity
of
Analy/cs
• Workflows
for
Impact
• Ac/ve
Learning

49. Digital
Humani9es
Roadmap
• Influence
Lora Aroyo
directly
Heritage
organiza/ons
• Interdisciplinary
research
with
computer
sciences
• Smart,
connected,
open
heritage
• Visible,
usable,
durable
heritage
• Shared
knowledge
infrastructure

50. Agora
Team:
Lora
Aroyo,
Guus
Schreiber,
Lourens
van
der
Meij,
Marieke
van
Erp,
Chiel
van
den
Akker,
Susan
Legêne,
Geertje
Jacobs,
Johan
Oomen

51. DIVE
Team:
Victor
de
Boer,
Oana
Inel,
Lora
Aroyo,
Johan
Oomen,
Elco
Van
Staveren,
Werner
Helmich
&
Dennis
De
Beurs
dive.beeldengeluid.nl

52. SealincMedia
Team:
Alessandro
Bozzon,
Geert-­‐Jan
Houben,
Lora
Aroyo,
Lizzy
Jongma,
Guus
Schreiber,
Chris
Dijkshoorn,
Jasper
Oosterman,
Jacco
van
Ossenbruggen,
Archana
NoMamkandath,
Myriam
Traub
http://sealinc.ops.few.vu.nl/invenit/

53. CrowdTruth
Team:
Lora
Aroyo,
Chris
Welty
Robert-­‐Jan
Sips,
Carlos
Mar9nez
Or9z,
Anca
Dumitrache,
Oana
Inel,
Benjamin
Timmermans,
Susanna
van
de
Ven,
Merel
van
Empel,
Jelle
v.d.
Ploeg,
Ta/ana
Cristea,
Khalid
Khamkham,
HarriëMe
Smook,
Rens
van
Honschooten,
Arne
Rutjes
CrowdTruth.org
github.com/CrowdTruth

54. Links
On
the
Web
• http://waida.nl
• http://prestoprime.org
• http://agora.cs.vu.nl
• http://sealincmedia.wordpress.com
• http://notube.tv
• http://vista-tv.eu
• http://dive.beeldengeluid.nl
• http://crowdtruth.org
• http://game.crowdtruth.org
• http://wm.cs.vu.nl
On
Twider
@waisda
@agora-­‐project
@sealincmedia
@prestocenter
@notubeproject
@vistatv
#CrowdTruth
http://lora-aroyo.org ! http://slideshare.net/laroyo ! @laroyo

55. lora-aroyo.org
slideshare.com/laroyo
@laroyo