Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Video Indexing And Retrieval


Published on

Published in: Technology, Business
  • Be the first to comment

Video Indexing And Retrieval

  1. 1. Video Indexing and Retrieval SLIS 5206
  2. 2. Text indexing Began in earnest after the printing press was invented in the 1400s. Scholarly journals began to be published with rapidity and indexing methods Were desperately needed. Pre-coordinate indexing Post-coordinate indexing Computerized indexing: KWIC—keyword in context String searches
  3. 3. Still image indexing <ul><li>James Turner, 1997: “Indexing Images, Some Considerations” </li></ul><ul><li>Images are subject to more than one interpretation; text is not </li></ul><ul><li>Text can stand alone; images rarely do </li></ul>
  4. 4. Video indexing <ul><li>Video indexing applications : news video, film archives, surveillance, user-generated content, distance learning, video conferencing, medical applications, sports. </li></ul><ul><li>Video indexing involves “segmentation, analysis and abstraction” of video content (Zhong). </li></ul>
  5. 5. Problems/Goals <ul><li>Growing amounts of video data </li></ul><ul><li>Video data is difficult to index ; dynamic not static. Ex: TV video has 25-30 frames per second. Bibliographic schemes: different manifestations of videos—languages, content added or edited, etc. </li></ul><ul><li>Copyright issues: not many videos are in public domain. </li></ul><ul><li>----------------------------------------------------------------------- </li></ul><ul><li>Goal = automated semantic indexing; not quite there yet. </li></ul>
  6. 6. Indexing breakdown <ul><li>Sequence->scene->shot->frame->object </li></ul><ul><li>Frame =still image </li></ul><ul><li>Key frame =representative still image </li></ul>
  7. 7. Types of indexing: low-level <ul><li>Based on color histograms, motion & object detection and tracking. </li></ul><ul><li>Focuses on appearance </li></ul><ul><li>Sequence->shot-> scene </li></ul><ul><li>Sequence =group of scenes </li></ul><ul><li>Scene =group of shots, similar to a paragraph in a text document. </li></ul><ul><li>Shot = “single series of actions with one camera.” </li></ul><ul><li>The basic unit of indexing, similar to a word in a text document. Scenes are similar to paragraphs in a text document while sequences are similar to pages or chapters. </li></ul>
  8. 8. Types of indexing: high-level <ul><li>High level indexing focuses on the content of the video, rather than its appearance </li></ul><ul><li>Semantic gap: the difference between description of content and how the user perceives the content. </li></ul><ul><li>Ex: content described by indexer as “boat”; user perceives it as “cruise” </li></ul><ul><li>Entities mentioned but not seen must still be indexed. Ex: newsreel of Bob Hope making joke about Marilyn Monroe (she may not actually appear in footage) </li></ul>
  9. 9. Scene cut detection algorithms (Zhong) <ul><li>1. Divide video streams into units (such as shots) </li></ul><ul><li>2. Select representative or KEY frame </li></ul><ul><li>3. Describe colors and shapes for indexing </li></ul><ul><li>Note: Temporal information not included </li></ul><ul><li>(must be separately annotated along with metadata) </li></ul>
  10. 10. Metadata <ul><li>Information that describes a resource; aids in classification </li></ul><ul><li>Metadata standards used for video indexing: </li></ul><ul><li>FIAF EAD SMIL MPEG-7 </li></ul><ul><li>Dublin Core XML RDF MPEG-21 </li></ul>
  11. 11. Metadata standards, cont’d <ul><li>FIAF —International Film Archive Federation cataloging rules </li></ul><ul><li>RDF —Resource Description Framework; uses subject-predicate-object descriptions </li></ul><ul><li>XML —eXtensible Markup Language—used for sharing data across different info systems </li></ul><ul><li>SMIL —Synchronized Multimedia Integration Language—an XML markup language for describing multimedia presentations </li></ul>
  12. 12. Metadata standards, cont’d <ul><li>MPEG-7 multimedia content description interface (Wiley 2002) </li></ul>
  13. 13. Metadata Standards cont’d <ul><li>Dublin Core (fifteen many elements, many qualifiers) can be applied to video indexing </li></ul><ul><li>E ncoded A rchival D escription —used for film archives; can be mapped to Dublin Core </li></ul><ul><li>MPEG-7 —mutimedia content description standard </li></ul><ul><li>MPEG-21 —Rights Expression Language, designed to discourage illegal file-sharing </li></ul>
  14. 14. Ex: Dublin Core and Video Indexing
  15. 15. Retrieval <ul><li>Granularity : how do users want to retrieve materials? i.e. segments, scenes, entire video? </li></ul><ul><li>Purpose : news, entertainment, business, security? </li></ul><ul><li>User expertise : technical users vs. general consumers </li></ul><ul><li>Database type </li></ul>
  16. 16. Ex: Content-based video retrieval and indexing model (Zhong, 2001)
  17. 17. Real-life example: News archives <ul><li>Special issues: they use large amounts of video daily and must archive them immediately. </li></ul><ul><li>“ On the fly” bibliographic schemes and indexing methods. </li></ul><ul><li>Ex: CNN uses its own cataloging scheme. </li></ul>
  18. 18. Online Film archives <ul><li>Internet Archive http:// /details/movies </li></ul><ul><li>British Film Institute http:// </li></ul><ul><li>Google Video (now includes film from National Archives) and </li></ul><ul><li>Youtube </li></ul><ul><li>International News Archives </li></ul><ul><li>Blinkx —over 7 million hours of video. </li></ul>
  19. 19. Current and Future Trends: User-generated content <ul><li>General public can upload/access videos through sites such as Youtube. </li></ul><ul><li>Retrieval is imprecise: often based on keywords, date and relevance. </li></ul><ul><li>Other resources: vlogs or video blogs. </li></ul><ul><li>User-generated content and demand for quick and precise access to content will continue to grow. </li></ul>