Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

BCS_Seminar.ppt

581 views

Published on

  • Be the first to comment

  • Be the first to like this

BCS_Seminar.ppt

  1. 1. Lexical Chains for Topic Detection and Tracking British Classification Society Feb 23rd 2001 Joe Carthy & Nicola Stokes University College Dublin [email_address] [email_address] http://www.cs.ucd.ie/staff/jcarthy Tel. +353 1 706 2481 or 706 2469 Fax. +353 1 269 7262
  2. 2. Topic Detection and Tracking <ul><li>Topic Detection and Tracking (TDT) </li></ul><ul><ul><li>DARPA funded TDT project with UMass, CMU and Dragon Systems </li></ul></ul><ul><ul><li>Domain is all broadcast news: written and spoken </li></ul></ul><ul><li>TDT includes: </li></ul><ul><ul><li>First story Detection </li></ul></ul><ul><ul><li>Event Tracking </li></ul></ul><ul><ul><li>Segmentation </li></ul></ul><ul><li>Applications </li></ul><ul><ul><li>digital news editors </li></ul></ul><ul><ul><li>media analysts </li></ul></ul><ul><ul><li>equity traders </li></ul></ul>
  3. 3. Topic Tracking and Detection <ul><li>Tracking may be defined as </li></ul><ul><ul><li>Take a corpus of news stories </li></ul></ul><ul><ul><li>Given 1 (or 2,4,8,16) sample stories about an event </li></ul></ul><ul><ul><li>Find all subsequent stories in the corpus about that event </li></ul></ul><ul><li>Detection: Is this a new story ? </li></ul>
  4. 4. <ul><li>Event is defined by a list of stories that discuss the event e.g. “ Kobe earthquake ” is defined by first story that describes this event </li></ul>Topic Tracking and Detection
  5. 5. UCD TDT ARCHITECTURE SERVER Lexical Chainer Event Tracker Event Detector
  6. 6. Topic Detection and Tracking DATE: 02:36 TITLE: O.J. SIMPSON Bought Knife, Murder Hearing told O.J. SIMPSON MURDER TRIAL NYC SUBWAY BOMBINGS CARLOS THE JACKEL DATA STREAM Previous Stories
  7. 7. <ul><li>Implemented Benchmark systems using conventional IR techniques: </li></ul><ul><ul><li>Stemmed keywords </li></ul></ul><ul><ul><li>Stopword removal(Porter) </li></ul></ul><ul><ul><li>Term weighting (Robertson, Sparck Jones) </li></ul></ul>Benchmark Systems
  8. 8. Lexical Chaining <ul><ul><li>Lexical chains - textual cohesion (Halliday & Hasan) </li></ul></ul><ul><ul><li>Cohesion: text makes sense as a whole </li></ul></ul><ul><ul><li>Cohesion occurs where the interpretation of one item is dependent of that of another item in the text. It is this dependency that gives rise to cohesion. </li></ul></ul>
  9. 9. Lexical Chaining <ul><ul><li>Where the cohesive elements occur over a number of sentences a cohesive chain is formed . </li></ul></ul><ul><ul><li>For example, the sentences: John had mud pie for dessert. Mud pie is made of chocolate. John really enjoyed it . </li></ul></ul><ul><ul><li>give rise to the lexical chain: {mud pie, dessert, mud pie, chocolate, it} </li></ul></ul><ul><ul><li>Lexical cohesion is as the name suggests lexical - it involves the selection of a lexical item that is in some way related to one occurring previously. </li></ul></ul>
  10. 10. Lexical Chaining <ul><ul><li>Reiteration is a form of lexical cohesion which involves the repetition of a lexical item. This may involve simple repetition of the word but also includes the use of a synonym , near-synonym or superordinate . For example in the sentences John bought a Jag. He loves the car. a superordinate, car , refers back to a subordinate Jag . The part-whole relationship is also an example of lexical cohesion e.g. airplane and wing . </li></ul></ul><ul><ul><li>A lexical chain is a sequence of related words in the text, spanning short or long distances. </li></ul></ul>
  11. 11. Lexical Chaining <ul><ul><li>A chain is independent of the grammatical structure of the text and in effect it is a list of words that captures a portion of the cohesive structure of the text. </li></ul></ul><ul><ul><li>A lexical chain can provide a context for the resolution of an ambiguous term and enable identification of the concept the term represents i.e. word sense disambiguation </li></ul></ul><ul><ul><li>Morris and Hirst were the first researchers to suggest the use of lexical chains to determine the structure of texts. </li></ul></ul>
  12. 12. Lexical Chaining <ul><ul><li>By identifying the lexical chains in a news story we hope to identify the focus of a news story. This can then be used in tracking and detection. </li></ul></ul><ul><ul><li>It is important to realise that determining lexical chains is not a sophisticated natural language analysis process. </li></ul></ul><ul><ul><li>Other Applications of Lexical Chaining </li></ul></ul><ul><ul><ul><li>Hypertext links: Green </li></ul></ul></ul><ul><ul><ul><li>Summarisation: Barzilay </li></ul></ul></ul><ul><ul><ul><li>Segmentation: Okumura and Honda </li></ul></ul></ul><ul><ul><ul><li>IR: Stairmand , Ellman , Mochizuki </li></ul></ul></ul><ul><ul><ul><li>Malapropism detection: St. Onge </li></ul></ul></ul><ul><ul><ul><li>Multimedia indexing: Kazman , Al-Halimi </li></ul></ul></ul>
  13. 13. Chain Generation <ul><ul><ul><ul><li>In order to construct lexical chains we must be able to identify relationships between terms. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>This is made possible by the use of WordNet </li></ul></ul></ul></ul><ul><ul><ul><ul><li>WordNet is a computational lexicon which was developed at Princeton University. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>In WordNet, synonym sets (synsets) are used to represent concepts where a synonym set corresponds to a concept and consists of all those terms that may be used to refer to that concept. </li></ul></ul></ul></ul>
  14. 14. Chain Generation <ul><ul><ul><ul><li>For example, take the concept airplane it is represented by the synset {airplane, aeroplane, plane}. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>A WordNet synset has a numerical identifier such as 02054514. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>Links between synsets in WordNet represent conceptual relations such as synonymy, hyponymy, meronymy (part-of) etc. </li></ul></ul></ul></ul><ul><ul><ul><ul><li>The synset identifier can be used to represent the concept referred to in the synset, for indexing and lexical chaining purposes. </li></ul></ul></ul></ul>
  15. 15. Word Sense Disambiguation 1st Term EXHAUST Part of Has a Term i CAR Car_exhaust 32748 Tire_out, Fatigue 374222 Train 3984 Exhaust 32748 Railway carriage 324932 Automobile 057643
  16. 16. Chain Generation <ul><li>Chaining procedure for a story: </li></ul><ul><ul><li>Take the i th term in the story and generate the set Neighbour i of its related synsets </li></ul></ul><ul><ul><li>For each other term, if it is a member of the set Neighbour i then add it to the lexical chain for term i . </li></ul></ul><ul><ul><li>If the lexical chain contains 3 or more elements then store the chain in a chain index file </li></ul></ul><ul><ul><li>Repeat above for all terms in the story. </li></ul></ul>
  17. 17. <ul><ul><li>Computing Chain_Sim(Trackset i , Story j ) </li></ul></ul><ul><ul><ul><li>Overlap Coefficient which may be defined as follows, for two lexical chains c 1 and c 2 : </li></ul></ul></ul><ul><ul><ul><li>Overlap Coefficient = </li></ul></ul></ul>
  18. 18. Evaluation Metrics <ul><ul><li>System returns a set of S documents : </li></ul></ul><ul><ul><ul><li>a = # in S discussing new events </li></ul></ul></ul><ul><ul><ul><li>b = # in S not discussing new events </li></ul></ul></ul><ul><ul><ul><li>c = # in S ' discussing new events </li></ul></ul></ul><ul><ul><ul><li>d = # in S ' not discussing new events </li></ul></ul></ul><ul><ul><li>Recall = a / (a+c) </li></ul></ul><ul><ul><li>Precision = a / (a+b) </li></ul></ul><ul><ul><li>Miss Rate = c / (a+c) = 1 - R </li></ul></ul><ul><ul><li>False Alarm Rate = b / (b+d) = Fallout </li></ul></ul>
  19. 19. Tracking Results
  20. 20. Tracking Results
  21. 21. Detection Results
  22. 22. Analysis of results <ul><ul><li>Expected trade-off between precision and recall </li></ul></ul><ul><ul><li>Small number of stories are sufficient to construct a tracking query </li></ul></ul><ul><ul><li>Performance in line with other TDT researchers </li></ul></ul><ul><ul><li>Lexical Chains - Improvement not significant ? </li></ul></ul>
  23. 23. TDT and Lexical Chain References <ul><li>Allan, J., Carbonell, J., Doddington, G., Yamron, J, and Yang, Y ., “Topic Detection and Tracking Pilot Study: Final Report”, Proceedings of the DARPA Broadcast News Transcription and Understanding Workshop , Morgan Kaufmann, San Francisco,1998. </li></ul><ul><li>Allan, J., Papka, R., and Lavrenko, V., “Online New Event Detection and Tracking”, Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , Melbourne, Australia, August 1998. </li></ul><ul><li>Barzilay, R., “Lexical Chains for Summarization ”, M.Sc. Thesis, Ben-Gurion University of the Negev , Israel, November 1997. </li></ul><ul><li>Barzilay, R., and Elhadad, M., “Using Lexical Chains for Text Summarization ”, The Fifth Bar-Ilan Symposium on Foundations of Artificial Intelligence Focusing on Intelligent Agents , Bar-Ilan University, Ramat Gan, Israel, June, 1997 </li></ul><ul><li>Budanitsky, A., “ Lexical Semantic Relatedness and its Application in Natural Language Processing”, (PhD thesis) Technical Report CSRG-390 , University of Toronto, 1999. </li></ul><ul><li>Ellman, J., “Using Roget's Thesaurus to Determine the Similarity of Texts”, PhD Thesis, University of Sunderland , 2000. </li></ul><ul><li>Fellbaum, C., (Ed.), WordNet: An Electronic Lexical Database and Some of its Applications , MIT Press, 1998. </li></ul><ul><li>Green, S.J., “Automatically Generating Hypertext by Computing Semantic Similarity”, Ph.D. Thesis, University of Toronto , 1997. </li></ul>
  24. 24. <ul><li>Halliday, M.A.K. and Hasan, R., “Cohesion In English”, Longman , 1976. </li></ul><ul><li>Hatch, P., &quot;Lexical Chaining for the Online Detection of New Events&quot;, M.Sc. Thesis, University College Dublin , 2000. </li></ul><ul><li>Hirst, G., and St-Onge, D., “Lexical Chains as Representations of Context for the Detection and Correction of Malapropisms”, in WordNet: An Electronic Lexical Database and Some of its Applications , Fellbaum, C., (Ed.), MIT Press, 1998. </li></ul><ul><li>Kazman, R., Al-Halimi, R., Hunt, W., and Mantei, M., “Four Paradigms for Indexing Video Conferences”, IEEE MultiMedia , 3 (1), Spring 1996. </li></ul><ul><li>Mochizuki, H., Iwayama, M., and Okumura, M., “Passage Level Document Retrieval Using Lexical Chains”, RIAO 2000, Content Based Multimedia Information Access , 491-506, 2000. </li></ul><ul><li>Morris J., and Hirst, G., “Lexical Cohesion, the Thesaurus, and the Structure of Text”, Computational Linguistics , 17 (1), 211-232, 1991. </li></ul><ul><li>Okumura, M., and Honda, T., “Word Sense Disambiguation and Text Segmentation Based on Lexical Cohesion”, In Proceedings of the Fifteenth International Conference on Computational Linguistics (COLING-94), Vol. 2, 775-761, Kyoto, Japan, August 1994. </li></ul><ul><li>Porter, M.F., “An Algorithm for Suffix Stripping”, Program , 14, 130-137, 1980. </li></ul><ul><li>Robertson, S.E. and Sparck Jones, K, &quot;Simple Approaches to Text Retrieval&quot;, University of Cambridge Computing Laboratory Technical Report Number 356 , May 1997. </li></ul><ul><li>Stairmand, M.A., “A Computational Analysis of Lexical Cohesion with Applications in Information Retrieval”, Ph.D. Thesis, UMIST , 1996. </li></ul><ul><li>Stokes, N., Carthy, J., First Story Detection using a Composite Document Representation, HLT 2001, Human Language Technology Confererence, San Diego, California, March 18-21, 2001 </li></ul><ul><li>TDT2000, “The Year 2000 Topic Detection and Tracking (TDT2000) Task Definition and Evaluation Plan”, available at the following URL: http://morph. ldc . upenn . edu /TDT/Guide/manual.front.html , November 2000. </li></ul>

×