Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • New business sectors emerged – aliances – convergence of different business sectors [each business sector + example – one by one in figure] [categories non mutually exclusive] From content owner to the end-user  those involved in the chain Of new technologies This justifies alliances in daily basis, acquisitions etc. Dependencies and dep trends (in using video search)
  • Reference to LREC06 paper Include slides from that ppt with examples + x-media indexing effects If no time, refer orally to what each detected relation means for an indexing mechanism Refer to tasks that need x-media decision mechanisms (indexing, categorization, summarization)
  • ppt

    1. 1. Crossing Media for Video Search: enabling usability beyond traditional broadcast & TV Katerina Pastra and Stelios Piperidis Language Technology Applications, Institute for Language and Speech Processing, Athens, Greece
    2. 2. The “Pervasive Digital Video” Era <ul><li>TV sets extended with “intelligent” DVRs, set-top boxes with PC-like functionalities, linked with PCs that display streamed video and allow interaction through gaming consoles </li></ul><ul><li>Video viewing transferred beyond the TV set, to mobiles & i-pods allowing on-the-move viewing </li></ul><ul><li>Video broadcast carried through broadband using IP </li></ul><ul><li>Video content (professional and/or consumer-generated) exchanged through file swapping and headline syndication technology </li></ul>
    3. 3. A New Era in Video Search ? From the digital video libraries context to the new pervasive digital video reality : The scope of video search (indexing & retrieval) technologies is broadened and their role is reinforced  Does pervasive digital video affect the “search” in video search technologies? (imposes new challenges)  Does video search affect the “pervasiveness” of digital video? (affects usability of available video and corresponding new technologies) 
    4. 4. Overview <ul><li>Video search: the market perspective </li></ul><ul><li>- market players and video search developers </li></ul><ul><li>- video search in commercial prototypes </li></ul><ul><li>Video search research prototypes </li></ul><ul><li>- lessons from the digital video library scenarios </li></ul><ul><li>New technological challenges </li></ul><ul><li>Dealing with new challenges </li></ul><ul><li>- Suggestions from the REVEAL THIS project </li></ul><ul><li>- Using cross-media decision mechanisms </li></ul>
    5. 5. Video Search & the Market Players Transitive Dependencies Dependency Trends Web content Aggregators, content service providers, content repackaging companies TV service providers & file-swapping networks Electronics Manufacturers Content owners IPTV software developers ISP, computer networking & phone companies Video search software developers
    6. 6. Video search mechanisms in the market <ul><li>Characteristics : </li></ul><ul><li>Use of owner/broadcaster created metadata </li></ul><ul><li>Text-based search on closed captions or </li></ul><ul><li>ASR or speech stream </li></ul><ul><li>Processing of English files mostly </li></ul><ul><li>Keyword query (restricted semantic expansion) </li></ul><ul><li>Retrieval unit is either whole video or short segment </li></ul><ul><li>where keyword appears (+ few seconds before and after) </li></ul>Are such mechanisms efficient? Quest for coherence Not robust Not always present Other languages? Find the right keyword problem!
    7. 7. Video Search in Research <ul><li>Lessons learned (Hauptmann and Christel 2004): </li></ul><ul><li>Fusion of medium-specific retrieval results boosts video retrieval performance slightly (vs. e.g. text-based only retrieval) </li></ul><ul><li>Fusion based on linear weights ~ query-type helpful </li></ul><ul><li>Text query/ASR enhancement, relevance feedback and feature-concept associations, all helpful </li></ul><ul><li>The digital video library access scenarios prevailed in research projects up until the ’00s (cf. Informedia, TRECVid etc.) </li></ul><ul><li>Video indexing & retrieval prototypes explored a variety of unimodal and multimodal mechanisms that go beyond commercially offered video search </li></ul>MM approaches slightly better. Necessary?
    8. 8. Video search challenges in the new context <ul><li>From advanced computer users to laymen </li></ul><ul><li>- type & quality of query </li></ul><ul><li>- expectations & requirements on retrieval accuracy </li></ul><ul><li>- length of retrieval unit </li></ul><ul><li>- domain and language of data </li></ul><ul><li>From structured data collections to pervasive video </li></ul><ul><li>- genre, domain, language, source/structure variation </li></ul><ul><li>- consumer-generated / noisy / low quality and prof. </li></ul><ul><li>- broadcast metadata, closed captions etc. availability </li></ul><ul><li>From static to dynamic search </li></ul><ul><li>- VoD & real-time broadcast data </li></ul><ul><li>- re-active & pro-active , personalised search </li></ul><ul><li>(push and pull) </li></ul><ul><li>The ideal search mechanism? </li></ul>
    9. 9. <ul><li>Multimedia approaches for video search suggested by research projects with application scenarios related to the new pervasive digital video context, e.g. leisure and entertainment in the digital home, and/or for the mobile user (e.g UP-TV, BUSMAN, AceMedia) </li></ul><ul><li>Image features – language concepts association for video search suggested (Multimedia integration) </li></ul><ul><li>REVEAL THIS goes a step further in suggesting the use of cross-media decision mechanisms </li></ul>Video search prototypes in the ’00s FP6 funded project (Nov. 2004-April 2007) http://www.reveal-this.org
    10. 10. REVEAL THIS Use Scenario <ul><li>A system that offer s both types of service : </li></ul><ul><li>Multim edia and Cross lingual Information Retrieval (pull) </li></ul><ul><li>Multimedia and Cross lingual information Filtering (push) </li></ul>Media Archive Search archive WEB Radio TV, Radio, Web data TV Reveal-THIS technology Mobile phone, and Web interfaces User User profile Mobile Web Local Archive Delivery EN-EL, European Parliament plenary sessions & press-conferences, national news, travel documentaries & info Media Server (Content Aggregator)
    11. 11. audio Media Manager SPC–speech processing cross-media stories text video FDIC - face Detection & identification TPC - text processing text keyframes Story Boundary Detection IAC – keyframe extraction, Image Analysis & image categorization Automatically extracted metadata: TPC: named entities, terms, facts Text Categories SPC: speaker turns, speaker names, text IAC: shotcuts, keyframes, image features, image categories FDIC: face regions, names Text categorisation Cross-media Indexing, Cross-media Categorization Multimedia Summarization Translation REVEAL This System Architecture Web text radio TV Media Server: Storage Personalisation Browse Query Retrieve Push Notifications
    12. 12. The notion of Cross-Media Decision Mechanisms <ul><li>Mechanisms that decide on the relation that holds between medium specific pieces of information: </li></ul><ul><li>across documents (Boll et al. 1999) </li></ul><ul><li>within documents (Pastra 2006) </li></ul><ul><li>The mechanisms decided whether medium-specific pieces of information within the same Multimedia Document are: </li></ul><ul><li>associated (multimedia integration) </li></ul><ul><li>complementary </li></ul><ul><li>semantically compatible/incompatible </li></ul>complementarity independence equivalence
    13. 13. Conclusions <ul><li>The scope of video search technology is broadened & new technological challenges are imposed </li></ul><ul><li>The market players consider video search technology indispensable </li></ul><ul><li>Commercial video search limited; research in the digital library access context goes beyond such limitations & points to slight benefits in using multimedia fusion techniques </li></ul><ul><li>Research with new application scenarios (iTV etc.) emphasizes the necessity of such mechanisms & introduces the notion of x-media decision mechanisms </li></ul>Efficient video search is indispensable for usability beyond traditional broadcast and TV; x-media decision mechanisms may hold the key for achieving it