Propbank Instance Annotation GuidelinesUsing a Dedicated Editor, JubileeJinho Choi, Claire Bonial, Martha PalmerInstitute ...
Upcoming SlideShare
Loading in …5

Propbank Instance Annotation Guidelines Using a Dedicated Editor, Jubilee


Published on

This paper gives guidelines of how to annotate Propbank instances using a dedicated editor, Jubilee. Propbank is a corpus in which the arguments of each verb predicate are annotated with their semantic roles in relation to the predicate. Propbank annotation also requires the choice of a sense ID for each predicate. Jubilee facilitates this annotation process by displaying several resources of syntactic and semantic information simultaneously: the syntactic structure of a sentence is displayed in the main frame, the available senses with their corresponding argument structures are displayed in another frame, all available Propbank arguments are displayed for the annotators choice, and example annotations of each sense of the predicate are available to the annotator for viewing. Easy access to each of these resources allows the annotator to quickly absorb and apply the necessary syntactic and semantic information pertinent to each predicate for consistent and efficient annotation. Jubilee has been successfully adapted to many Propbank projects in several universities. The tool runs platform independently, is light enough to run as an X11 application and supports multiple languages such as Arabic, Chinese, English, Hindi and Korean.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Propbank Instance Annotation Guidelines Using a Dedicated Editor, Jubilee

  1. 1. Propbank Instance Annotation GuidelinesUsing a Dedicated Editor, JubileeJinho Choi, Claire Bonial, Martha PalmerInstitute of Cognitive Science, University of Colorado at Boulder<br />Propbank<br />Propbank Annotation Procedure<br />Jubilee<br />Advantages and Features<br />More about Jubilee<br />Acknowledgements<br /><ul><li> A corpus in which the arguments of each verb predicate are annotated with their semantic roles.
  2. 2. Each predicate is also annotated with its sense id.
  3. 3. Annotations are done over syntactic trees.
  4. 4. Each task is claimed, double-annotated, and adjudicated.
  5. 5. In the past, three different tools were used:</li></ul>to claim tasks.<br />to annotate arguments. <br />to annotate verb senses.<br />Operators<br /><ul><li> In the absence of Treebank co-indexing, annotators can provide semantic information about a null element by manually linking it to its overt referent using the ‘★’ operator.
  6. 6. In the cases where an argument is discontinuous such that it cannot be captured in the annotation of one node, the ‘,’ operator is used.
  7. 7. The ‘&’ operator is used to link the object trace after a passive verb to its referent in the subject position in reduced relative clauses.</li></ul>How to obtain Jubilee<br /><ul><li> Available as an open source project on Google code (
  8. 8. Contact:
  9. 9. Special thanks are due to Professor NianwenXue of Brandeis University for his very helpful insights, as well as Scott Cotton, the developer of RATS and Tom Morton, the developer of WordFreak, both previously used for Propbank annotation.
  10. 10. We also gratefully acknowledge the support of the National Science Foundation Grants CISE-CRI-0551615, Towards a Comprehensive Linguistic Annotation and CISE- CRI 0709167, Collaborative: A Multi-Representational and Multi-Layered Treebank for Hindi/Urdu, and a grant from the Defense Advanced Research Projects Agency (DARPA/IPTO) under the GALE program, DARPA/CMO Contract No. HR0011-06-C-0022, subcontract from BBN, Inc. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.</li></ul>Frameset view<br /><ul><li> Displays and allows annotators to choose the sense (roleset) ofthe predicate with respect to thecurrent tree.</li></ul>Task 1<br />Task 2<br />Lemma of the predicatefor the selected roleset<br />List of roleset IDsfor the predicate<br />View examplesof the selected roleset <br />Adjudication 2<br />Annotation 2-1<br />Annotation 2-2<br />Double-annotate<br />Claim<br />Adjudicate<br />…<br />arg0 (agent)<br />2<br />1<br />3<br />Jubilee<br />arg1 (theme)<br />Sense: open.01<br />α<br />arg2 (instrument)<br />A definition and a generalized argument structure of the selected roleset<br />Claiming tasks<br /><ul><li> Choose a Propbank project.
  11. 11. Choose a task from either:
  12. 12. New tasks: claimed by one or less annotator.
  13. 13. My tasks: claimed by the current annotator.</li></ul>Treebank view in annotation mode<br /><ul><li> Displays syntactic trees in the selected task.</li></ul>Jubilee main window<br />List of tree IDs<br />Annotator ID<br />Navigation buttons<br />Raw sentence of the tree<br />Argument view<br /><ul><li> Contains buttons representing Propbank argument labels.</li></ul>Treebank view in adjudication mode<br /><ul><li> Displays and allows adjudicators to choose or edit from multiple annotations.</li></ul>Multiple annotations<br />Adjudicator ID<br />Annotation vs. adjudication mode<br /><ul><li> In annotation mode, annotators are allowed to view and edit only tasks claimed by themselves or one other annotator.
  14. 14. In adjudication mode, adjudicators are allowed to view and edit all tasks that have undergone at least single-annotation.
  15. 15. Platform independent: runs on any platform with JVM (Java 6.0).
  16. 16. Multilingual: accommodates Arabic, Chinese, English, Hindi and Korean.
  17. 17. Run on X11: annotators can make updates remotely.
  18. 18. Syntax visualization: syntax is easily understandable to annotators.
  19. 19. Semantic supply: frameset info is provided to consult annotators.
  20. 20. Speed up: argument and sense annotations are simultaneous.
  21. 21. Unified format: the use of one tool simplifies data maintenance.</li>