Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Television in Words TIWO Round Table EPSRC GR/R67194/01 Softel 18 th September 2002
  2. 2. “TIWO News” <ul><li>Visit from Prof. James Turner , School of Library and Information Science, University of Montreal </li></ul><ul><li>Contact from local company, Force10 - supplier of Low Vision Aids and Assistive hearing products ; and from Surrey Association for Visual Impairment </li></ul><ul><li>Andrew Vassiliou – PhD student starts October </li></ul><ul><li>Mike Graham – MSc student, has started </li></ul>
  3. 3. “TIWO News” <ul><li>Papers presented: </li></ul><ul><ul><li>LREC 2002 Workshop on Temporal Information in Natural Language </li></ul></ul><ul><ul><li>TKE 2002 , Terminology and Knowledge Engineering, ‘Words for Pictures: analysing a corpus of art texts’ </li></ul></ul><ul><ul><li>Banff New Media Institute , workshop on AI and New Media ‘Narrative in Multimedia Systems’ </li></ul></ul>
  4. 4. “TIWO News” <ul><li>VACE – Video Analysis and Content Exploitation , Advanced Research and Development Activity (ARDA) </li></ul><ul><ul><li>automatic content detection and recognition technologies for two primary video data sources: video scenes of various indoor and outdoor activities involving people, meetings, and vehicles, and TV news broadcasts. </li></ul></ul><ul><ul><ul><li>(1) indexing and retrieval for video data; </li></ul></ul></ul><ul><ul><ul><li>(2) autonomous video understanding; </li></ul></ul></ul><ul><ul><ul><li>(3) ancillary improvement for still image processing; </li></ul></ul></ul><ul><ul><ul><li>(4) enabling technologies for video data mining, filtering and selection; and </li></ul></ul></ul><ul><ul><ul><li>(5) a drastic reduction in volume for video storage and forwarding mechanisms. </li></ul></ul></ul><ul><li> </li></ul>
  5. 5. “TIWO News” <ul><li>UniS GRID Project Proposal </li></ul><ul><ul><li>Proposal for a GRID ‘Centre of Excellence’ at Surrey: an infrastructure to support future projects </li></ul></ul><ul><ul><li>Focus on language based information and knowledge access on the GRID </li></ul></ul><ul><ul><li>Future projects may include “TIWO 2” (alongside projects in the areas of finance; criminal investigation; digital heritage; medical images, etc.) </li></ul></ul><ul><ul><li>Currently inviting organisations to express support and register interest in future projects </li></ul></ul>
  6. 6. Summary of Progress <ul><li>Corpus Building and Analysis </li></ul><ul><li>System Development </li></ul><ul><li>“Narrative” – reading group over the summer </li></ul>
  7. 7. “Defining” Narrative <ul><li>“ a primary resource for structuring and comprehending experience”; “a discourse style and a cognitive style ”; “realised in combinations of media” </li></ul><ul><li>“ a sequence of (causally) connected events, organised in space and time” </li></ul><ul><li>“ usually the agents of cause and effect are characters” </li></ul><ul><li>“ audience creates a richly represented fictional world” </li></ul><ul><li>“ viewer recalls information, anticipates what will follow, infers events not explicitly mentioned / depicted” </li></ul><ul><li>“ narrative comprehension involves mental stores and inferences in relation to: text-specific knowledge, world knowledge and knowledge of genre” </li></ul>
  8. 8. “Computing” Narrative <ul><li>Video data models tend to comprise entities, events, actions and spatio-temporal relations; may with to add AI to deal with further aspects of narrative… </li></ul><ul><li>Knowledge-bases for text-specific and world knowledge, including stereotypical situations </li></ul><ul><li>Representing characters “psychological drives” </li></ul><ul><li>Representing and reasoning about intentions / emotions </li></ul><ul><li>Maintaining belief models and perspectives; viewer, machine and characters </li></ul>
  9. 9. Cautionary Note <ul><li>“ More than reconstructed timelines and inventories of existents , storyworlds are mentally and emotionally projected environments in which interpreters are called upon to live out complex blends of cognitive and imaginative response , encompassing sympathy, the drawing of causal inference, identification, evaluation, suspense, and so on” </li></ul><ul><li>David Herman, Story Logic (2002). </li></ul>
  10. 10. Analysis of “Narrative Features” in a Corpus of Audio Description Scripts <ul><li>Focussed on emotive states by observing occurrences of words associated with emotional states in audio description scripts, e.g. JOY ( happy, happily, pleasure, contentedly ), DISTRESS ( miserably, sadly ), FEAR ( anxiously, desperately ), etc. </li></ul><ul><li>Resulting graphs characterise changing emotional states during a film… </li></ul>
  11. 11. Corpus Building and Analysis Elia Tomadaki
  12. 12. Corpus linguistics and narrative <ul><li>Any collection of more than one text can be called a corpus, the Latin equivalent for “body”. Thus, a corpus is any body of text. In the context of modern linguistics, it appears to have four basic characteristics: Sampling and representativeness, finite size, machine-readable form and a standard reference. Corpus linguistics deal with the study and use of language through corpora. </li></ul><ul><li>Linguists analyse corpora of narrative discourse and have observed features such as frequent reference to perfect aspect, third person reference etc. Therefore, this area of study is interesting for an AD corpus. </li></ul>
  13. 13. Corpus building 244,100 TOTAL 26,400 4 Documentaries 22,500 5 Children’s programmes 24,000 4 Recipes 17,600 8 Series 153,600 24 Films Num of words Num of scripts Type
  14. 14. GATE system
  15. 15. English Patient AD: A comparison Num of words Describer/s - Company 31,560 (approx. 1,500 dialogue) Saul Zaentz - Saul Zaentz Company 7,436 Di Langford - RNIB 6,736 Louise Fryer and Michael Baker – ITFC
  16. 16. Most frequent words 15 10 8 pilot 223 21 33 Patient 267 77 63 Katharine 340 74 81 Almasy 368 73 73 Hana Saul Zaentz Company frequency ITFC frequency RNIB frequency Word
  17. 17. An example Suddenly an explosion shatters the calm as the jeep runs over a mine (13 words) the jeep explodes in a ball of flame. (8 words) An explosion on the road ahead. The jeep has hit a mine (12 words) Saul Zaentz - Saul Zaentz Company Louise Fryer and Michael Baker - ITFC Di Langford - RNIB
  18. 18. System Development Yan Xu
  19. 19. Aims and objectives <ul><li>Be able to browse, index video data based on inferences about the semantic content </li></ul><ul><li>Make the machine “ understand ” the story --narrative </li></ul><ul><li>Knowledge representation: build up general knowledge( CYC, Commonsense) and text-specific knowledge </li></ul>
  20. 20. Film Film Editing Scene Title Sequence End-credits Shot Text Audio Description Dialogue Narrative Time Prop Location Character Event State Inferred Events Explicitly Event Non-diegetic Plot Plot Explicitly Event Non-diegetic Plot
  21. 21. Text Audio Description Dialogue Film Film Editing Scene Title Sequence End-credits Shot Narrative Time Prop Location Character Event State Inferred Events Plot Non-diegetic Plot Explicitly Event
  22. 22. Text Film Narrative Event Inferred Events Plot Non-diegetic Plot Explicitly Event
  23. 23. Feedback and any questions?