Nicole Nogoy at the #FORCE2016 Annotating All Knowledge Coalition Face-to-Face Kick Off Meeting: Community Annotation in the GigaCuration Challenge

1. Community Annotation in the
Giga-Curation Challenge
Annotating All Knowledge Coalition Face-to-Face Kick Off
Meeting
April 17, 2016
Nicole Nogoy
Commissioning Editor, GigaScience
Nicole@gigasciencejournal.com

2. Why Community Curation?
• Curation and annotation – under appreciated parts of scholarship
• Extra information for efficient and easy re-use
• Improve curation quantity and utility by accurate addition of essential metadata
• Impossible task for one person or group
• A need for easy and efficient curation:
• Tons of data across multiple repositories and need to link, collate and share
peoples interpretations via
• All talk and no action

3. The Giga-Curation Challenge 2016
• Use case – curator wanted to study how many people community
curation can reach out to via Hypothes.is and iCLiKVAL and the
outcome
• Challenge held at ISB BioCuration2016, April 10-14th
• Attendees encouraged to annotate data in GigaDB (and beyond) via
Hypothes.is and iCLiKVAL general purpose annotation tools
• Monitored by the Annotometer – with live leaderboard
Code is in GitHub: https://github.com/gigascience/annotometer

4. Lessons Learned
• Slow to get started – so encourage pre-registration next time

5. Lessons Learned
• Slow to get started – so encourage pre-registration next time
• Each resource needs an introduction
• People seem shy about sharing annotations publicly
• There is a need for a dedicated time slot to give everyone a chance to
use the tools.

6. Feedback
• People did not understand what was expected of them
• Next time- get a bigger time slot for introductions
• The goal was too vague
• Next time- give specific tasks to get them started
• And provide a better target (e.g. make X annotations and get a T-shirt)
• A barrier to joining for some was the fact they had to register for 3 separate
accounts
• Next time- try to make it a single registration to create all accounts
• Hold shorter competitions within the main one (one day competitions)

7. Results
• Number of Annotometer registered users = 21 (~10% of attendees)
• iCLikVAL = 8 reg. users made annotations
• Hypothes.is = 10 reg. users made annotations
• Total number of annotations made = 1201
• Cost of annotations =
cost of prize/ No annotations = $0.41 USD per annotation

8. And the winner is…..

9. Show of Hands
• If we run it again next year (with actions from all the lessons we’ve
learned this year), who would join in?
• If any other tools want to join in (PhenPackets?), get in touch.
• If you want to run your own challenge using the Annotometer, get in
touch. (its all open-source:https://github.com/gigascience/annotometer)
For feedback - please email, Chris: chris@gigasciencejournal.com

10. Thanks – to everyone that took part and HUGE thank you
to the conference organisers
the Annotometer Hypothes.is iCLiKVAL
Jesse Xiao
Maryann Martone
And
thanks to
Todd TaylorNaveen Kumar