A Corpus-based Study of Temporal Signals

636 views

Published on

Automatic temporal ordering of events described in discourse has been of great interest in recent years. Event orderings are
conveyed in text via various linguistic mechanisms including the use of expressions such as “before”, “after” or “during”
that explicitly assert a temporal relation – temporal signals. We investigate the role of temporal signals in temporal relation extraction and provide a quantitative analysis of these expressions in the TimeBank annotated corpus.

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
636
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

A Corpus-based Study of Temporal Signals

  1. 1. Introduction Temporal links Temporal signals Improving annotation Summary A Corpus-based Study of Temporal Signals Leon Derczynski University of Sheffield 20 July, 2011Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  2. 2. Introduction Temporal links Temporal signals Improving annotation SummaryOutline 1 Introduction 2 Temporal links 3 Temporal signals 4 Improving annotation 5 SummaryLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  3. 3. Introduction Temporal links Temporal signals Improving annotation SummaryMotivation Language for time helps us describe: changes planning history Time is not always explicit in natural language – we don’t include a timestamp with every action Goals: Try to automatically extract temporal information from documents, so that we can build a model that connects information in a text with timeLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  4. 4. Introduction Temporal links Temporal signals Improving annotation SummaryTemporal Entities What elements can we try to extract from discourse? Each document might contain: Basic primitives: Events – occurences, states, reports Times – dates and times, durations, sets Linkages between primitives: general temporal link aspectual links and subordination We can use the basic primitives as nodes on a graph, and links as its arcs.Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  5. 5. Introduction Temporal links Temporal signals Improving annotation SummaryOutline 1 Introduction 2 Temporal links 3 Temporal signals 4 Improving annotation 5 SummaryLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  6. 6. Introduction Temporal links Temporal signals Improving annotation SummaryTemporal link labelling How do we label the links between temporal entities? First, choose a relation set: TimeML gives us 13, including before, simultaneous, includes.. Some relations have transitive and commutative properties: If “a before b” and “b before c” then we can infer “a before c” This means that consistency can be important Develop a gold-standard corpus – TimeBankLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  7. 7. Introduction Temporal links Temporal signals Improving annotation SummaryAutomated temporal link labelling How can we automatically label links? Machine learning approaches: teach ourselves how to label a link based on times and events it may connect Use TimeBank and other as examples of how A difficult task: notable research effort, including various evaluation exercises, have attempted it Overall accuracy remains around 60% – 70% : too low1 1 See Chambers & Jurafsky, 2008; Mirroshandel et. al. 2010; TempEval-2010Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  8. 8. Introduction Temporal links Temporal signals Improving annotation SummarySource of temporal linking information What information can we use to label links? If a human can manage to understand temporal relations, the information must be somewhere Possible sources: – tense and aspect – world knowledge – discourse structure – specific time information (at 9 o’clock) – explicit signals: temporal conjunctionsLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  9. 9. Introduction Temporal links Temporal signals Improving annotation SummaryOutline 1 Introduction 2 Temporal links 3 Temporal signals 4 Improving annotation 5 SummaryLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  10. 10. Introduction Temporal links Temporal signals Improving annotation SummaryTemporal conjunctions Are these words/phrases useful for automatic understanding? A baseline system could learn to label links with 62% accuracy With simple modification, links in TimeBank that had associated signals could be annotated with 83% accuracy Clear indication that signals are an accessible source of temporal informationLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  11. 11. Introduction Temporal links Temporal signals Improving annotation SummaryTemporal conjunctions in newswire What do temporal conjunctions look like in TimeBank? 11.2% of temporal links are annotated as having one (718 instances) Top words: – prepositions (in, for, on) – conjunctions (after, before, since)Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  12. 12. Introduction Temporal links Temporal signals Improving annotation SummaryTemporal conjunctions in newswire Occurrences Likelihood of Phrase Corpus freq. as signal being a signal subsequently 3 3 100% after 72 67 93% follows 4 3 75% before 33 23 70% until 36 25 69% during 19 13 68% as soon as 3 2 67% Table: A sample of phrases most likely to be annotated as a signal when they occur in TimeBank, which occur more than once in the corpus.Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  13. 13. Introduction Temporal links Temporal signals Improving annotation SummaryDiscrimination of temporal signal words What else are these temporal signal words used for? Some words are very likely to have a temporal sense: subsequently – 3 instances, all temporal; after – 72 instances, 93% temporal. Other words are versatile: from – 366 instances, 5% temporal. between – 33 instances, 1 temporal;Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  14. 14. Introduction Temporal links Temporal signals Improving annotation SummarySignal-to-link relations What temporal relations do these words signify? after doesn’t always signify a temporal after relation Word order is important After I ate, I went to bed I ate after I went to bed Signal phrase TimeML relation Frequency after AFTER 56 after ENDS 6 after BEGINS 4 after IAFTER 1 already BEFORE 6 already INCLUDES 4 already IS INCLUDED 3Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  15. 15. Introduction Temporal links Temporal signals Improving annotation SummarySignal class How can we characterise temporal signals? Signals are likely to belong to a closed class of words Common prepositions as seen earlier Some adverbs – previously, subsequently Set phrases – as soon as, so farLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  16. 16. Introduction Temporal links Temporal signals Improving annotation SummarySpatial/Temporal overlap Time and space are related and events are constrained in terms of both Language for space and time has some similarities before has both temporal and spatial senses Spatially annotated corpora – SpatialML Relative spatial links in this corpus are much more likely to employ a signal (97.5%) Possible explanation – temporal language is more diverse (tense, auxiliaries)Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  17. 17. Introduction Temporal links Temporal signals Improving annotation SummaryOutline 1 Introduction 2 Temporal links 3 Temporal signals 4 Improving annotation 5 SummaryLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  18. 18. Introduction Temporal links Temporal signals Improving annotation SummaryRe-annotation Are these signals correctly annotated in TimeBank? Manual examination: start with words that are likely to be temporal signals before: found 33 times in the corpus, 23 are signals Many under-annotated cases: before the war began was scheduled to return to port before hostilities eruptedLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  19. 19. Introduction Temporal links Temporal signals Improving annotation SummaryRe-annotation How could we improve signal annotation? Linguistic description of temporal conjunctions may be weak Annotation guidelines may be insufficient Solution: provide an enhanced signal description, and revise TimeBank accordinglyLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  20. 20. Introduction Temporal links Temporal signals Improving annotation SummaryFormal signal description A temporal signal is a word that indicates the type of temporal relation between two intervals Signal surface forms have a head and an optional quantifier shortly after – quantified temporal signal Temporal signals have exactly two arguments (events and/or times) One argument may be implicit (e.g. for Later)Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  21. 21. Introduction Temporal links Temporal signals Improving annotation SummaryAugmented TimeBank We examined 30 of the most frequent signal words and phrases that were not annotated as temporal This comprised around 1 000 instances in text We annotated any missed temporal signals, including EVENT and TLINK annotations where required This resulted in 15.8% of TLINKs using a signalLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  22. 22. Introduction Temporal links Temporal signals Improving annotation SummaryOutline 1 Introduction 2 Temporal links 3 Temporal signals 4 Improving annotation 5 SummaryLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  23. 23. Introduction Temporal links Temporal signals Improving annotation SummaryConclusion Temporal signals are a usable and important source of information We have provided a definition for temporal signals Existing corpora have been upgraded with better annotationLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  24. 24. Introduction Temporal links Temporal signals Improving annotation SummaryFuture work Automatic signal discrimination Signal association Applying findings to spatial languageLeon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals
  25. 25. Introduction Temporal links Temporal signals Improving annotation Summary Thank you. Are there any questions?Leon Derczynski University of SheffieldA Corpus-based Study of Temporal Signals

×