Video Indexing And Retrieval
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Video Indexing And Retrieval






Total Views
Views on SlideShare
Embed Views



3 Embeds 17 13 3 1



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Video Indexing And Retrieval Presentation Transcript

  • 1. Video Indexing and Retrieval SLIS 5206
  • 2. Text indexing Began in earnest after the printing press was invented in the 1400s. Scholarly journals began to be published with rapidity and indexing methods Were desperately needed. Pre-coordinate indexing Post-coordinate indexing Computerized indexing: KWIC—keyword in context String searches
  • 3. Still image indexing
    • James Turner, 1997: “Indexing Images, Some Considerations”
    • Images are subject to more than one interpretation; text is not
    • Text can stand alone; images rarely do
  • 4. Video indexing
    • Video indexing applications : news video, film archives, surveillance, user-generated content, distance learning, video conferencing, medical applications, sports.
    • Video indexing involves “segmentation, analysis and abstraction” of video content (Zhong).
  • 5. Problems/Goals
    • Growing amounts of video data
    • Video data is difficult to index ; dynamic not static. Ex: TV video has 25-30 frames per second. Bibliographic schemes: different manifestations of videos—languages, content added or edited, etc.
    • Copyright issues: not many videos are in public domain.
    • -----------------------------------------------------------------------
    • Goal = automated semantic indexing; not quite there yet.
  • 6. Indexing breakdown
    • Sequence->scene->shot->frame->object
    • Frame =still image
    • Key frame =representative still image
  • 7. Types of indexing: low-level
    • Based on color histograms, motion & object detection and tracking.
    • Focuses on appearance
    • Sequence->shot-> scene
    • Sequence =group of scenes
    • Scene =group of shots, similar to a paragraph in a text document.
    • Shot = “single series of actions with one camera.”
    • The basic unit of indexing, similar to a word in a text document. Scenes are similar to paragraphs in a text document while sequences are similar to pages or chapters.
  • 8. Types of indexing: high-level
    • High level indexing focuses on the content of the video, rather than its appearance
    • Semantic gap: the difference between description of content and how the user perceives the content.
    • Ex: content described by indexer as “boat”; user perceives it as “cruise”
    • Entities mentioned but not seen must still be indexed. Ex: newsreel of Bob Hope making joke about Marilyn Monroe (she may not actually appear in footage)
  • 9. Scene cut detection algorithms (Zhong)
    • 1. Divide video streams into units (such as shots)
    • 2. Select representative or KEY frame
    • 3. Describe colors and shapes for indexing
    • Note: Temporal information not included
    • (must be separately annotated along with metadata)
  • 10. Metadata
    • Information that describes a resource; aids in classification
    • Metadata standards used for video indexing:
    • Dublin Core XML RDF MPEG-21
  • 11. Metadata standards, cont’d
    • FIAF —International Film Archive Federation cataloging rules
    • RDF —Resource Description Framework; uses subject-predicate-object descriptions
    • XML —eXtensible Markup Language—used for sharing data across different info systems
    • SMIL —Synchronized Multimedia Integration Language—an XML markup language for describing multimedia presentations
  • 12. Metadata standards, cont’d
    • MPEG-7 multimedia content description interface (Wiley 2002)
  • 13. Metadata Standards cont’d
    • Dublin Core (fifteen many elements, many qualifiers) can be applied to video indexing
    • E ncoded A rchival D escription —used for film archives; can be mapped to Dublin Core
    • MPEG-7 —mutimedia content description standard
    • MPEG-21 —Rights Expression Language, designed to discourage illegal file-sharing
  • 14. Ex: Dublin Core and Video Indexing
  • 15. Retrieval
    • Granularity : how do users want to retrieve materials? i.e. segments, scenes, entire video?
    • Purpose : news, entertainment, business, security?
    • User expertise : technical users vs. general consumers
    • Database type
  • 16. Ex: Content-based video retrieval and indexing model (Zhong, 2001)
  • 17. Real-life example: News archives
    • Special issues: they use large amounts of video daily and must archive them immediately.
    • “ On the fly” bibliographic schemes and indexing methods.
    • Ex: CNN uses its own cataloging scheme.
  • 18. Online Film archives
    • Internet Archive http:// /details/movies
    • British Film Institute http://
    • Google Video (now includes film from National Archives) and
    • Youtube
    • International News Archives
    • Blinkx —over 7 million hours of video.
  • 19. Current and Future Trends: User-generated content
    • General public can upload/access videos through sites such as Youtube.
    • Retrieval is imprecise: often based on keywords, date and relevance.
    • Other resources: vlogs or video blogs.
    • User-generated content and demand for quick and precise access to content will continue to grow.