Lora Aroyo @laroyo http://lora-aroyo.org
CATCHing up with People
from individual access to crowd engagement
Lora Aroyo
Lora Aroyo @laroyo http://lora-aroyo.org
… and more
Lora Aroyo @laroyo http://lora-aroyo.org
2004 - 2014
CHOICE
STITCH
SCRATCH
MITCH
CHIP
RICH
Lora Aroyo @laroyo http://lora-aroyo.org
2004 - 2014
HiTiME
MuNCH
CHOICE
STITCH
SCRATCH
MITCH
CHIP
RICH
BRIDGE
MuSEUM
Agora
LINKS
CHORAL
WITCHCRAFT
Lora Aroyo @laroyo http://lora-aroyo.org
… life after CATCH
COMMIT/
SealincMedia
NLeSC
DIVE
DIVE+
CLARIN
Polimedia
CLARIAH
….
Lora Aroyo @laroyo http://lora-aroyo.org
before CATCH
Lora Aroyo @laroyo http://lora-aroyo.org
Museums, Libraries & Archives
as Inventories of the World
André	
  Malraux,	
  The	
  Imaginary	
  Museum	
  of	
  World	
  Sculpture,	
  1953	
  
Lora Aroyo @laroyo http://lora-aroyo.org
Museums, Libraries & Archives
as a Place to Engage with the World
Lora Aroyo @laroyo http://lora-aroyo.org
Digital Content at Web Scale
Lora Aroyo @laroyo http://lora-aroyo.org
Bridging the Gap between
Digital & Physical
Lora Aroyo @laroyo http://lora-aroyo.org
Bridging the Gap between
Past & Present
Lora Aroyo @laroyo http://lora-aroyo.org
Transforming Cultural Heritage …
Lora Aroyo @laroyo http://lora-aroyo.org
… into engaging experience
Lora Aroyo @laroyo http://lora-aroyo.org
… into personal experience
Lora Aroyo @laroyo http://lora-aroyo.org
Empowering End-Users
Lora Aroyo @laroyo http://lora-aroyo.org
Enabling (Digital) Humanities
Lora Aroyo @laroyo http://lora-aroyo.org
Expanding Computer Science
Lora Aroyo @laroyo http://lora-aroyo.org
Project: CHIP
Lora Aroyo @laroyo http://lora-aroyo.org
Enriching Collection Metadata
Lora Aroyo @laroyo http://lora-aroyo.org
Creator:
Rembrandt van Rijn
Lora Aroyo @laroyo http://lora-aroyo.org
Creator:
Rembrandt van Rijn
Lora Aroyo @laroyo http://lora-aroyo.org
Style:
Baroque
Lora Aroyo @laroyo http://lora-aroyo.org
Subject:
Portraiture
Lora Aroyo @laroyo http://lora-aroyo.org
Creation Site:
Amsterdam
Lora Aroyo @laroyo http://lora-aroyo.org
1642
Rembrandt
The Night Watch
Militia
Oil paint
Baroque
creationDate
Creator
subject
material
Amsterdam
creationSite
Ferdinand BolPieter Lastman
studentOf
North Holland
Portraiture
subject
Male portraits
specific termpart of
teacherOf
style
Wang,	
  Y.,	
  Stash,	
  N.,	
  Aroyo,	
  L.,	
  Gorgels,	
  P.,	
  Rutledge,	
  L.	
  and	
  Schreiber,	
  G.,	
  Recommenda>ons	
  Based	
  on	
  Seman>cally-­‐enriched	
  Museum	
  Collec>ons,	
  
Journal	
  of	
  Web	
  Seman>cs:	
  Science,	
  Services	
  and	
  Agents	
  on	
  the	
  World	
  Wide	
  Web,	
  Volume	
  6,	
  Issue	
  4,	
  Pages	
  43	
  -­‐	
  50,	
  November,	
  2008	
  
Lora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.org
style: Baroque
teacher of: Gerrit Dou
teacher of: Nicolaes Maesteacher of: Ferdinand Bol
self-portrait
militia
place:
Amsterdam period:
1625 to 1650
Lora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.org
CHIP
Semantics-driven Recommendations
Lora Aroyo @laroyo http://lora-aroyo.org
Semantics-driven Museum Tours
Lora Aroyo @laroyo http://lora-aroyo.org
Personalized Mobile Guide
Lora Aroyo @laroyo http://lora-aroyo.org
Project: AGORA
Lora Aroyo @laroyo http://lora-aroyo.org
Metadata Enrichment with Events
Lora Aroyo @laroyo http://lora-aroyo.org
Erp,	
  M.	
  van;	
  Oomen,	
  J.;	
  Segers,	
  R.;	
  Akker,	
  C.	
  van	
  de;	
  Aroyo,	
  L.;	
  Jacobs,	
  G.;	
  Legêne,	
  S;	
  Meij,	
  L.	
  van	
  der;O	
  ssenbruggen,	
  J.R.	
  van;	
  Schreiber,	
  G.	
  
Automa>c	
  Heritage	
  Metadata	
  Enrichment	
  with	
  Historic	
  Events	
  Museums	
  and	
  the	
  Web	
  2011	
  
Lora Aroyo @laroyo http://lora-aroyo.org
engaging
users
through
event
narratives
Lora Aroyo @laroyo http://lora-aroyo.org
DIGITAL HERMENEUTICS
theory of interpretation: relation parts of wholes
events as context for interpretation of online collections
intersection of hermeneutics & Web technology
“Digital Hermeneutics: Agora and the online understanding of cultural heritage” In proceedings of Web Science Conference,
(ACM: New York, 2011)
Lora Aroyo @laroyo http://lora-aroyo.org
Project: DIVE
Lora Aroyo @laroyo http://lora-aroyo.org
support a multitude of perspectives
Lora Aroyo @laroyo http://lora-aroyo.org
Simple Event Model (SEM)

OpenAnnotation (OA) & SKOS
DIVE:MEDIA OBJECT SEM:EVENT
SEM:PLACE
SEM:TIME
SEM:ACTOR
SKOS:CONCEPT
OA:ANNOTATION
• LINKS TO EUROPEANA (MULTILINGUAL)
• LINKS TO DBPEDIA
Lora Aroyo @laroyo http://lora-aroyo.org
OPENIMAGES.EU
• 3000 videos 
• NL Institute for Sound & Vision
• mostly news broadcasts
DELPHER.NL
•  1.5 Million Scans
•  Radio bulletins
•  (hand annotated)
•  1937 – 1984
Lora Aroyo @laroyo http://lora-aroyo.org
En0ty	
  &	
  Event	
  Extra0on	
  with	
  CrowdTruth.org	
  
ENTITY EXTRACTION
EVENTS CROWDSOURCING AND LINKING TO
CONCEPTS THROUGH CROWDTRUTH.ORG
SEGMENTATION & KEYFRAMES
LINKING EVENTS AND 
CONCEPTS TO KEYFRAMES
Lora Aroyo @laroyo http://lora-aroyo.org
Crowdsourcing	
  Video	
  Analysis	
  
CrowdTruth.org
Lora Aroyo @laroyo http://lora-aroyo.org
Crowdsourcing	
  Text	
  Analysis	
  
CrowdTruth.org
Lora Aroyo @laroyo http://lora-aroyo.org
Crowdsourcing Diversity of
Perspectives
Lora Aroyo @laroyo http://lora-aroyo.org
http://crowdtruth.org
Lora Aroyo @laroyo http://lora-aroyo.org
dive.beeldengeluid.nl
3rd	
  Price	
  at	
  the	
  Seman>c	
  Web	
  Challenge	
  2014	
  
Lora Aroyo @laroyo http://lora-aroyo.org
Project: SealincMedia
Lora Aroyo @laroyo http://lora-aroyo.org
expertise of museum professionals is
in annotating their collection
with art-historical information, e.g. when they
were created, by whom, etc.
Lora Aroyo @laroyo http://lora-aroyo.org
detailed domain-specific information
about depicted objects, e.g. which species the
animal or plant belongs to,
is in most cases not available
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Related	
  
Recommenda0ons	
  
Lora Aroyo @laroyo http://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
Serendipity	
  
Lora Aroyo @laroyo http://lora-aroyo.org
Structured Semantics for 
CLUSTERING, RANKING, TRUST & INTEREST
Lora Aroyo @laroyo http://lora-aroyo.org
Engage	
  the	
  Niche	
  in	
  Cultural	
  Heritage	
  
http://annotate.accurator.nl
Lora Aroyo @laroyo http://lora-aroyo.org
use nichesourcing, i.e. niches of people with the
right expertise, to add more specific information
Lora Aroyo @laroyo http://lora-aroyo.org
Engage	
  with	
  Games	
  in	
  Cultural	
  Heritage	
  
training the general crowd to be a niche:
game in which players can carry out an expert
annotation tasks with some assistance
Lora Aroyo @laroyo http://lora-aroyo.org
Engage	
  with	
  Games	
  in	
  Cultural	
  Heritage	
  
training the general crowd to be a niche:
game in which players can carry out an expert
annotation tasks with some assistance
Lora Aroyo @laroyo http://lora-aroyo.org
data enrichment
interpretation support
capturing multiple perspectives
crowd & nichesourcing
Power to the People
from the People
Lora Aroyo @laroyo http://lora-aroyo.org
… and more
Lora Aroyo @laroyo http://lora-aroyo.org
… and more
Lora Aroyo @laroyo http://lora-aroyo.org
Lora Aroyo @laroyo http://lora-aroyo.org
http://chip.win.tue.nl/
http://agora.cs.vu.nl
http://dive.beeldengeluid.nl
http://crowdtruth.org
http://annotate.accurator.nl
http://wm.cs.vu.nl

Keynote @Final NWO CATCH Program Event

  • 1.
    Lora Aroyo @laroyohttp://lora-aroyo.org CATCHing up with People from individual access to crowd engagement Lora Aroyo
  • 2.
    Lora Aroyo @laroyohttp://lora-aroyo.org … and more
  • 3.
    Lora Aroyo @laroyohttp://lora-aroyo.org 2004 - 2014 CHOICE STITCH SCRATCH MITCH CHIP RICH
  • 4.
    Lora Aroyo @laroyohttp://lora-aroyo.org 2004 - 2014 HiTiME MuNCH CHOICE STITCH SCRATCH MITCH CHIP RICH BRIDGE MuSEUM Agora LINKS CHORAL WITCHCRAFT
  • 5.
    Lora Aroyo @laroyohttp://lora-aroyo.org … life after CATCH COMMIT/ SealincMedia NLeSC DIVE DIVE+ CLARIN Polimedia CLARIAH ….
  • 6.
    Lora Aroyo @laroyohttp://lora-aroyo.org before CATCH
  • 7.
    Lora Aroyo @laroyohttp://lora-aroyo.org Museums, Libraries & Archives as Inventories of the World André  Malraux,  The  Imaginary  Museum  of  World  Sculpture,  1953  
  • 8.
    Lora Aroyo @laroyohttp://lora-aroyo.org Museums, Libraries & Archives as a Place to Engage with the World
  • 9.
    Lora Aroyo @laroyohttp://lora-aroyo.org Digital Content at Web Scale
  • 10.
    Lora Aroyo @laroyohttp://lora-aroyo.org Bridging the Gap between Digital & Physical
  • 11.
    Lora Aroyo @laroyohttp://lora-aroyo.org Bridging the Gap between Past & Present
  • 12.
    Lora Aroyo @laroyohttp://lora-aroyo.org Transforming Cultural Heritage …
  • 13.
    Lora Aroyo @laroyohttp://lora-aroyo.org … into engaging experience
  • 14.
    Lora Aroyo @laroyohttp://lora-aroyo.org … into personal experience
  • 15.
    Lora Aroyo @laroyohttp://lora-aroyo.org Empowering End-Users
  • 16.
    Lora Aroyo @laroyohttp://lora-aroyo.org Enabling (Digital) Humanities
  • 17.
    Lora Aroyo @laroyohttp://lora-aroyo.org Expanding Computer Science
  • 18.
    Lora Aroyo @laroyohttp://lora-aroyo.org Project: CHIP
  • 19.
    Lora Aroyo @laroyohttp://lora-aroyo.org Enriching Collection Metadata
  • 20.
    Lora Aroyo @laroyohttp://lora-aroyo.org Creator: Rembrandt van Rijn
  • 21.
    Lora Aroyo @laroyohttp://lora-aroyo.org Creator: Rembrandt van Rijn
  • 22.
    Lora Aroyo @laroyohttp://lora-aroyo.org Style: Baroque
  • 23.
    Lora Aroyo @laroyohttp://lora-aroyo.org Subject: Portraiture
  • 24.
    Lora Aroyo @laroyohttp://lora-aroyo.org Creation Site: Amsterdam
  • 25.
    Lora Aroyo @laroyohttp://lora-aroyo.org 1642 Rembrandt The Night Watch Militia Oil paint Baroque creationDate Creator subject material Amsterdam creationSite Ferdinand BolPieter Lastman studentOf North Holland Portraiture subject Male portraits specific termpart of teacherOf style Wang,  Y.,  Stash,  N.,  Aroyo,  L.,  Gorgels,  P.,  Rutledge,  L.  and  Schreiber,  G.,  Recommenda>ons  Based  on  Seman>cally-­‐enriched  Museum  Collec>ons,   Journal  of  Web  Seman>cs:  Science,  Services  and  Agents  on  the  World  Wide  Web,  Volume  6,  Issue  4,  Pages  43  -­‐  50,  November,  2008  
  • 26.
    Lora Aroyo @laroyohttp://lora-aroyo.org
  • 27.
    Lora Aroyo @laroyohttp://lora-aroyo.org style: Baroque teacher of: Gerrit Dou teacher of: Nicolaes Maesteacher of: Ferdinand Bol self-portrait militia place: Amsterdam period: 1625 to 1650
  • 28.
    Lora Aroyo @laroyohttp://lora-aroyo.org
  • 29.
    Lora Aroyo @laroyohttp://lora-aroyo.org CHIP Semantics-driven Recommendations
  • 30.
    Lora Aroyo @laroyohttp://lora-aroyo.org Semantics-driven Museum Tours
  • 31.
    Lora Aroyo @laroyohttp://lora-aroyo.org Personalized Mobile Guide
  • 32.
    Lora Aroyo @laroyohttp://lora-aroyo.org Project: AGORA
  • 33.
    Lora Aroyo @laroyohttp://lora-aroyo.org Metadata Enrichment with Events
  • 34.
    Lora Aroyo @laroyohttp://lora-aroyo.org Erp,  M.  van;  Oomen,  J.;  Segers,  R.;  Akker,  C.  van  de;  Aroyo,  L.;  Jacobs,  G.;  Legêne,  S;  Meij,  L.  van  der;O  ssenbruggen,  J.R.  van;  Schreiber,  G.   Automa>c  Heritage  Metadata  Enrichment  with  Historic  Events  Museums  and  the  Web  2011  
  • 35.
    Lora Aroyo @laroyohttp://lora-aroyo.org engaging users through event narratives
  • 36.
    Lora Aroyo @laroyohttp://lora-aroyo.org DIGITAL HERMENEUTICS theory of interpretation: relation parts of wholes events as context for interpretation of online collections intersection of hermeneutics & Web technology “Digital Hermeneutics: Agora and the online understanding of cultural heritage” In proceedings of Web Science Conference, (ACM: New York, 2011)
  • 37.
    Lora Aroyo @laroyohttp://lora-aroyo.org Project: DIVE
  • 38.
    Lora Aroyo @laroyohttp://lora-aroyo.org support a multitude of perspectives
  • 39.
    Lora Aroyo @laroyohttp://lora-aroyo.org Simple Event Model (SEM)
 OpenAnnotation (OA) & SKOS DIVE:MEDIA OBJECT SEM:EVENT SEM:PLACE SEM:TIME SEM:ACTOR SKOS:CONCEPT OA:ANNOTATION • LINKS TO EUROPEANA (MULTILINGUAL) • LINKS TO DBPEDIA
  • 40.
    Lora Aroyo @laroyohttp://lora-aroyo.org OPENIMAGES.EU • 3000 videos • NL Institute for Sound & Vision • mostly news broadcasts DELPHER.NL •  1.5 Million Scans •  Radio bulletins •  (hand annotated) •  1937 – 1984
  • 41.
    Lora Aroyo @laroyohttp://lora-aroyo.org En0ty  &  Event  Extra0on  with  CrowdTruth.org   ENTITY EXTRACTION EVENTS CROWDSOURCING AND LINKING TO CONCEPTS THROUGH CROWDTRUTH.ORG SEGMENTATION & KEYFRAMES LINKING EVENTS AND CONCEPTS TO KEYFRAMES
  • 42.
    Lora Aroyo @laroyohttp://lora-aroyo.org Crowdsourcing  Video  Analysis   CrowdTruth.org
  • 43.
    Lora Aroyo @laroyohttp://lora-aroyo.org Crowdsourcing  Text  Analysis   CrowdTruth.org
  • 44.
    Lora Aroyo @laroyohttp://lora-aroyo.org Crowdsourcing Diversity of Perspectives
  • 45.
    Lora Aroyo @laroyohttp://lora-aroyo.org http://crowdtruth.org
  • 46.
    Lora Aroyo @laroyohttp://lora-aroyo.org dive.beeldengeluid.nl 3rd  Price  at  the  Seman>c  Web  Challenge  2014  
  • 47.
    Lora Aroyo @laroyohttp://lora-aroyo.org Project: SealincMedia
  • 48.
    Lora Aroyo @laroyohttp://lora-aroyo.org expertise of museum professionals is in annotating their collection with art-historical information, e.g. when they were created, by whom, etc.
  • 49.
    Lora Aroyo @laroyohttp://lora-aroyo.org detailed domain-specific information about depicted objects, e.g. which species the animal or plant belongs to, is in most cases not available
  • 50.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 51.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 52.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 53.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 54.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 55.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 56.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org
  • 57.
    Lora Aroyo @laroyohttp://lora-aroyo.org
  • 58.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org Related   Recommenda0ons  
  • 59.
    Lora Aroyo @laroyohttp://lora-aroyo.orgLora Aroyo @laroyo http://lora-aroyo.org Serendipity  
  • 60.
    Lora Aroyo @laroyohttp://lora-aroyo.org Structured Semantics for CLUSTERING, RANKING, TRUST & INTEREST
  • 61.
    Lora Aroyo @laroyohttp://lora-aroyo.org Engage  the  Niche  in  Cultural  Heritage   http://annotate.accurator.nl
  • 62.
    Lora Aroyo @laroyohttp://lora-aroyo.org use nichesourcing, i.e. niches of people with the right expertise, to add more specific information
  • 63.
    Lora Aroyo @laroyohttp://lora-aroyo.org Engage  with  Games  in  Cultural  Heritage   training the general crowd to be a niche: game in which players can carry out an expert annotation tasks with some assistance
  • 64.
    Lora Aroyo @laroyohttp://lora-aroyo.org Engage  with  Games  in  Cultural  Heritage   training the general crowd to be a niche: game in which players can carry out an expert annotation tasks with some assistance
  • 65.
    Lora Aroyo @laroyohttp://lora-aroyo.org data enrichment interpretation support capturing multiple perspectives crowd & nichesourcing Power to the People from the People
  • 66.
    Lora Aroyo @laroyohttp://lora-aroyo.org … and more
  • 67.
    Lora Aroyo @laroyohttp://lora-aroyo.org … and more
  • 68.
    Lora Aroyo @laroyohttp://lora-aroyo.org
  • 69.
    Lora Aroyo @laroyohttp://lora-aroyo.org http://chip.win.tue.nl/ http://agora.cs.vu.nl http://dive.beeldengeluid.nl http://crowdtruth.org http://annotate.accurator.nl http://wm.cs.vu.nl