Crowdsourcing Knowledge-Intensive Tasks In Cultural Heritage: SealincMedia Accurator demonstrator @WebSci2014

2,375 views

Published on

Slides prepared by Jasper Oosterman (j.e.g.oosterman@tudelft.nl). Find more information at our website: http://sealincmedia.wordpress.com

Large datasets such as Cultural Heritage collections require detailed annotations when digitised and made available online. Annotating dierent aspects of such collections requires a variety of knowledge and expertise which is not always possessed by the collection curators. Artwork annotation is an example of a knowledge intensive image annotation task, i.e. a task that demands annotators to have domain-specic knowledge in order to be successfully completed.

Today, Lora Aroyo will present WebSci2014 conference the results of a study aimed at investigating the applicability of crowdsourcing techniques to knowledge intensive image annotation tasks. We observed a clear relationship between the annotation difficulty of an image, in terms of number of items to identify and annotate, and the performance of the recruited workers. Here you can see the poster and the slides of the presentation.

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
2,375
On SlideShare
0
From Embeds
0
Number of Embeds
1,322
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Crowdsourcing Knowledge-Intensive Tasks In Cultural Heritage: SealincMedia Accurator demonstrator @WebSci2014

  1. 1. Crowdsourcing  Knowledge-­‐Intensive  Tasks  In   Cultural  Heritage   J.  Oosterman,  A.  Bozzon,  G.J.  Houben   A.  NoCamkandath,  C.  Dijkshoorn,  L.  Aroyo  
  2. 2. AnnotaIon  of  digiIzed  Cultural     Heritage  with  specific  elements.   For  example:  Flowers,  Birds,  Castles  
  3. 3. Botanical  name  depicted  element   Rosa  banksiae   What  is  the  relaIon  between  enIty   idenIficaIon  difficulty  and  crowd  annotaIon   behavior?  
  4. 4. Complica0ons   Varying  amount  of  elements,  different  sizes   and  prominence.  Possibly  overlapping  or   lacking  detail.  
  5. 5. Examples  of  prints  in  the  collecIon  
  6. 6. AnnotaIon  task  template   Number  of  flowers   Number  of  flower  types   Up  to  three  flower  labels   Certainty  of  the  answers   References  
  7. 7. # Unable # Fantasy # Flower Name #Prints 1 10 20 30 40 Worker ID 0 10 20 30 40 #  Workers  opened  task   732   #  Workers  passed  test  quesIons   84   #  Selected  workers   44   #  AnnotaIon  tasks  performed   488   #  “Fantasy”  task   58   #  “Unable”  task   70   #  “Flowers”  task     360   #  Flower  labels   465   Tasks  performed  by  workers   LiCle  agreement  on     “unable”  or  “fantasy”   Tasks  requiring  domain     knowledge  not  popular    
  8. 8. # Instances # Prints 10 100 R ose Lilium Tulip Sunflow er D ianthus B ellis Iris Viola O rchidaceaePaeonia Papaver Lotus B ellis Perennis N arcissus H yacinth %WrongAnswer 0 0.2 0.4 0.6 0.8 1.0 # of Flowers NP P Flower Types NP P Flower  idenIficaIon  not   dependent  on  prominence     Most  workers  provide  simple  common  names,   but  also  workers  with  excepIonal  detail.  
  9. 9. Local  idenIficaIon   Outlook   Local  labeling  
  10. 10. hCp://sealincmedia.wordpress.com     sealincmedia-­‐all@lists.tudelj.nl       J.  Oosterman,  A.  Bozzon,  G.J.  Houben   A.  NoCamkandath,  C.  Dijkshoorn,  L.  Aroyo  

×