Accurator ask the right crowd, enrich your collection
Rijksmuseum Amsterdam holds an enormous collection which comprises over 1 million artworks
however, only a small fraction of about 8000 items are currently on display
to grant the public access to the objects in archives and depots, the Rijksmuseum started to digitize the artworks ...
… and present the collection online. 125.000 artworks are already available, and another 40.000 are added every year
the expertise of museum professionals lies in describing & annotating collection with arthistorical information, thus for ...
“We’re adding 40.000 items to the collection every year. After the scan, we have limited time for each painting and this o...
detailed information about the depicted objects, e.g. which species the animal or plant belongs to, is in most cases not a...
the need for more detailed annotations: this painting is annotated only with “bird with blue head near branch with red lea...
by involving people from outside the museum in annotation process, we support museum professionals in their annotation tas...
we use crowdsourcing to get more annotations. we use nichesourcing, i.e. niches of people with the right expertise, to add...
first, we use the crowd from Crowdflower & Amazon Mechanical Turk to make a general classification of the artworks
the crowd tags artworks on a generic level, e.g. ‘bird’, ‘flower’. Most people can provide common knowledge tags, but it is...
to fill this gap, we target experts
we use sources like Twitter to find experts or groups of experts on certain areas, e.g. bird lovers, ornithologists or peop...
these experts can contribute their knowledge about bird species using the Accurator platform
We create user profiles for each expert to better match the annotation tasks with the right expert, e.g. if an expert knows...
… she will be asked to annotate more of the former
we have developed a platform where users can enter tags, either by using terms from a structured vocabulary or by adding f...
experts can enter any information about the depicted object & they can also review the tags that others have provided
for tasks that are too difficult, we developed a game in which players can carry out an expert annotation task with some ass...
… and the possibility to gain points, compete with others keeps the users engaged
to evaluate the correctness of annotations they are reviewed & rated by other experts who have expertise in the same topic
to evaluate the correctness of annotations they are reviewed & rated by other experts who have expertise in the same topic
for example, if expert A has annotated the bird with Minivet and expert B, whose specialty is also Japanese birds, is cert...
next to the peer reviews, we use trust algorithms to determine the reputation of experts over time. This reputation is als...
Trust-aware Ranking & Relevance x Legend Accepted Tags Rejected Tags Cluster Medoid Reviewers Evaluate Evaluated Tags P...
we use the computed correctness to select high quality annotations
these annotations can serve two purposes:
1) refine the description of collection items 2) Fuel semantic search techniques
with such clever steps, we involve the social crowd of people in the annotation & curation of the museum’s vast collection
THANK YOU!
Upcoming SlideShare
Loading in …5
×

SealincMedia Accurator Demos

6,794 views

Published on

Finding relevant multimedia content is notoriously difficult, and the difficulty increases with the size and heterogeneity of the content collection. Linked cultural media collections are heterogeneous by nature and rapidly increase in size, mainly through enormous amounts of user-generated content and metadata that are placed on the Internet on a daily basis. Without mechanisms for keeping any part of these collections easily accessible by any user at any time and any use context, the value of these collections for the community will drop, just like their value as an economic asset.

demo: http://2-dot-rma-accurator.appspot.com/#Intro
website: http://sealincmedia.wordpress.com/

Published in: Technology, Sports
License: CC Attribution-NonCommercial-ShareAlike License
0 Comments
9 Likes
Statistics
Notes
no profile picture user

  • Be the first to comment

Show More
No Downloads
Views
Total views
6,794
On SlideShare
0
From Embeds
0
Number of Embeds
4,689
Actions
Shares
0
Downloads
7
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

SealincMedia Accurator Demos

  1. 1. Accurator ask the right crowd, enrich your collection
  2. 2. Rijksmuseum Amsterdam holds an enormous collection which comprises over 1 million artworks
  3. 3. however, only a small fraction of about 8000 items are currently on display
  4. 4. to grant the public access to the objects in archives and depots, the Rijksmuseum started to digitize the artworks ...
  5. 5. … and present the collection online. 125.000 artworks are already available, and another 40.000 are added every year
  6. 6. the expertise of museum professionals lies in describing & annotating collection with arthistorical information, thus for most artworks, we know when they were created, by whom
  7. 7. “We’re adding 40.000 items to the collection every year. After the scan, we have limited time for each painting and this occasionally results in incomplete annotations.” Henrike Hövelmann, Head of Print Cabinet Online
  8. 8. detailed information about the depicted objects, e.g. which species the animal or plant belongs to, is in most cases not available
  9. 9. the need for more detailed annotations: this painting is annotated only with “bird with blue head near branch with red leaf”, and the species of the bird and the plant are missing
  10. 10. by involving people from outside the museum in annotation process, we support museum professionals in their annotation task
  11. 11. we use crowdsourcing to get more annotations. we use nichesourcing, i.e. niches of people with the right expertise, to add more specific information
  12. 12. first, we use the crowd from Crowdflower & Amazon Mechanical Turk to make a general classification of the artworks
  13. 13. the crowd tags artworks on a generic level, e.g. ‘bird’, ‘flower’. Most people can provide common knowledge tags, but it is unlikely that they also know the scientific name of the bird.
  14. 14. to fill this gap, we target experts
  15. 15. we use sources like Twitter to find experts or groups of experts on certain areas, e.g. bird lovers, ornithologists or people who enjoy bird-watching in their spare time
  16. 16. these experts can contribute their knowledge about bird species using the Accurator platform
  17. 17. We create user profiles for each expert to better match the annotation tasks with the right expert, e.g. if an expert knows well songbirds, but not much of birds of prey …
  18. 18. … she will be asked to annotate more of the former
  19. 19. we have developed a platform where users can enter tags, either by using terms from a structured vocabulary or by adding free text
  20. 20. experts can enter any information about the depicted object & they can also review the tags that others have provided
  21. 21. for tasks that are too difficult, we developed a game in which players can carry out an expert annotation task with some assistance
  22. 22. … and the possibility to gain points, compete with others keeps the users engaged
  23. 23. to evaluate the correctness of annotations they are reviewed & rated by other experts who have expertise in the same topic
  24. 24. to evaluate the correctness of annotations they are reviewed & rated by other experts who have expertise in the same topic
  25. 25. for example, if expert A has annotated the bird with Minivet and expert B, whose specialty is also Japanese birds, is certain that this is not Minivet , he can rate expert A’s annotation as incorrect and add his own
  26. 26. next to the peer reviews, we use trust algorithms to determine the reputation of experts over time. This reputation is also considered when assessing the annotation correctness
  27. 27. Trust-aware Ranking & Relevance x Legend Accepted Tags Rejected Tags Cluster Medoid Reviewers Evaluate Evaluated Tags Provide External Annotators Extrapolate Provenance Un-evaluated Tags Generate Provenancebased estimates Tags x x x Generate x x + x Cluster semantically similar tags. Store the corresponding evidence. Reputationbased estimates Tag1 - Accept Tag2 - Reject Tag3 - Accept …… TagN - Accept Predict Tag evalutation
  28. 28. we use the computed correctness to select high quality annotations
  29. 29. these annotations can serve two purposes:
  30. 30. 1) refine the description of collection items 2) Fuel semantic search techniques
  31. 31. with such clever steps, we involve the social crowd of people in the annotation & curation of the museum’s vast collection
  32. 32. THANK YOU!

×