Your SlideShare is downloading. ×
Audiovisual Content Exploitation at FIA 15042010 NISV
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Audiovisual Content Exploitation at FIA 15042010 NISV


Published on

Published in: Technology

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide
  • The concept of technology-enhanced audiovisual indexing may have been a holy grail in earlier research, but recently it has become an urgent prerequisite in the context of our present-day information society. The effective exploitation of our Digital Libraries however, is currently impeded despite a wealth of technological progress. It becomes increasingly apparent that there may be an underlying problem, rooted in the disparity between technology and user needs. In order to respond to the demands of the information society advanced techniques and new workflow practices of use need to be explored by focusing on the needs of contemporary users, both professionals and non-professionals. At the same time, we need to foster opportunities for drawing user communities into the Digital Libraries, to involve them in enhancing content exploitability, e.g., by community tagging or capturing user generated content from the internet and aligning this with archived items. In this talk, the area of tension between the current state of technology and user needs is discussed in more detail in the context of practical use cases within the Netherlands Institute for Sound and Vision.
  • Let me first introduce Sound and Vision. Sound and Vision maintains and provides access to 70 per cent of the Dutch audio-visual heritage comprising approximately 700,000 hours of television, radio, music and film, making SV one of the largest audiovisual archives in Europe. The archive is growing. SV is the business archive of the national broadcasting corporationsand digitally born television and radio made by the Dutch public broadcastering companies is flowing in right after it is broadcast. Also, via the SV archive service PROARCHIVE content from Dutch cultural heritage institutes and regional broadcast institutes flows in. Finally we are currently selecting manually Dutch user generated content from the internet. More on this later.
  • € 173 mln in 7 jaar (looptijd 2007-2014)wel terugverdienverplichting. Tijdens looptijd € 19 mln
  • Om dit soort use cases mogelijk te maken is handmatig beschrijven van materiaal niet de oplossing want te kostbaar en per definitie beperkt. Bij contentbeheerders ontbreken vaak de resources om zelfs minimale beschrijvingen zoals titel en datum te produceren
  • Om materiaal toch, en in meer detail te kunnen beschrijven zijn een aantal strategieen denkbaar.Zo wordt de hulp van het publiek ingeroepen om materiaal te beschrijven, het zogenaamde crowdsourcing.Een belangrijk alternatief waar we in Twente onderzoek naar gedaan hebben is het gebruik van beschikbare tekstuele bronnen zoals ondertitels, notulen of aantekeningen, als beschrijvingen van content. Met behulp van spraaktechnologie kunnen deze beschrijvingen soms ook worden gesynchroniseerd met het materiaal (denk hierbij aan notulen) of is er al een vorm van synchronisatie aanwezig (in het geval van ondertitels. Maar vaak zijn dit soort bronnen niet beschikbaar. Dan kan technologie die op basis van audiovisuele kenmerken automatisch beschrijvingen genereert uitkomst bieden.
  • allemaal potentieel interessante content voor verschillende typen gebruikers:producers die uit zijn op hergebruik, journalisten, onderzoekers, en natuurlijk ook het algemeen publiek.Zomaar een greep uit mogelijke use cases waar je aan kunt denken:
  • Transcript

    • 1. Audiovisual content exploitation in the networked information society
      Roeland Ordelman
      Netherlands Institute for Sound and Vision
    • 2. contents
      • NISV context: Images of the Future
      • 3. “Access”, an important keyword in the business models ...
      • 4. ... but what about access in practice
      • 5. Technology and user interaction: from a ‘laboratory view’ on users to drawing them into the development chain
    • NISV context
      • +700.000 hours of radio, television, documentaries, films and music, over 2 million photographs, 20.000 objects like cameras, televisions, radios, costumes and pieces of scenery.
      • 6. still growing:
      • 7. digitally born television and radio programs made by the Dutch public broadcasting companies (video: 15K/hours/year)
      • 8. PROARCHIVE: archiving service
      • 9. selection of (Dutch) user generated content
    • 10. Images of the Future
      • Selection, restoration, digitization, encoding and storage of 137,000 hours of video, 20,000 hours of film, 124,000 hours of audio and more than three million photographs.
      • 11. One of the largest digitisation effort in Europe
      • 12. Three goals:
      • 13. Safeguarding heritage for future generations
      • 14. Creating social- economical value (“unlock the social and economic potential of the collections”)
      • 15. Innovation: new infrastructure for strengthening knowledge economy
      • 16. To achieve these objectives, the cultural heritage sector is challenged to re-evaluate its business models
    • Business model
      • The total investment of this initiative sums up to 173 million Euro
      • 17. A strong business model is necessary to support this kind of investment and prove that such an investment will result in long-term socio-economic returns
      • 18. The outcome of a Cost-Benefit analysis was positive: “The total balance of costs and returns of restoring, preserving and digitising audio-visual material (excluding costs of tax payments) will be between: 20+ and 60+ million.’’
      • 19. Economic benefits:
      • 20. Direct effects of the investment are revenues from sales, access for specific user groups, the repartition of copyright for the use of the material and so on.
      • 21. The indirect effects concern the product markets and labour market.
      • 22. Social benefits:
      • 23. conservation of culture, reinforcement of cultural awareness, reinforcement of democracy through the accessibility of information, increase in multimedia literacy and contribution to the Lisbon goals set by the EU
    • 24. Content exploitation: from content is king ...
    • 25. ... to metadata rules
    • 26. Manual annotation
      costly & limited
    • 27. Research on automatic annotation
      • automatic information extraction based on:
      • 28. visual features
      • 29. information from audio
      • 30. crowdsourcing
      • 31. deploying collateral data sources:
      • 32. subtitles, production scripts, meeting minutes, slides
    • Various (laboratory) showcases
      Commercial systems (e.g., blinkx, google)
      Progress? Yes!
    • 33. work in progress
      • institutional: reorganisation of traditional archival workflows
      • 34. national: development of common services
      • 35. OAI, Persistent Identifiers, ASR service, Vocabulary Repositories
      • 36. commercial: uptake by MNCs (Google and Microsoft) and SMEs
      • 37. individual: bring about a shift regarding defensive attitude of content owners towards opening up their funded and protected archives (trust/reliability)
    • Automatic annotation
      • Participation in international research projects
      • 38. VidioActive, MultiMATCH, VIDI-video, LiWA, P2P-Fusion, Sterna, EUScreen, PrestoPrime
      • 39. Collaboration agreement with Dutch research institutes
      • 40. Researchers stationed at Sound and Vision
      • 41. Provide data (TRECVID, VideoCLEF)
      • 42. Research environment: exact copy of iMMix production environment for testing new technology
      • 43. speech recognition
      • 44. video analysis
      • 45. fingerprinting
      • 46. linking of context data (web, program guide, production data)
    • Annotation strategies
      • crowdsourcing: video labeling game
      • 47. deploying collateral data sources: incorporation of subtitles
      • 48. automatic information extraction: speech recognition for radio, pilots with visual
      • 49. technology aided manual annotation: documentalist support
      • 50. linking to other information sources
    • media professionals
      general public
      disparity between technology and user needs
    • 51. Users perspective
      • Rapidly evolving networked information society
      • 52. Opening up
      • 53. Focus on community specific requirements
      • 54. search needs
      • 55. presentation/interaction needs
      • 56. Draw communities into libraries
    • Digital Archive
      Images for the Future
      >250.000 hrs of audio and video
      Digital Born
      15.000 hours of video
      40.000 hours of radio
      Asset management
      User generated content and metadata
      Public Web Acces
    • 57. "ifitdoesn't spread, it is dead" (Jenkins, 2009)
    • 58. Open Images
      • Open media platform for online access to audiovisualarchivematerial, availablefor free (creative) reuse
      • 59. Built by Sound and Vision & Knowledgeland
      • 60. Contributersinclude:
    • Open, open, open
      • Open source media platform (MMBase)
      • 61. Use of and open video codec (OggTheora)
      • 62. Use of the HTML5 <video> tag
      • 63. Use of an open API (OAI-PMH, Atomfeeds)
    • Licence
      • CC-BY-SA as preferredlicense
      • 64. 3,000 items fromour ‘own’ collection
      • 65. ‘Internet quality’
    • Open Images
      Rightsownedby Sound and Vision
      Digitised items
      Sound and Visioncollection
    • 66.
    • 67.
    • 68.
    • 69.
    • 70. community specific requirements
      From document level search to fragment level search
    • 71. 28
      Broadcast professionals
      In: Huurnink, Hollink, van Den Heuvel 2009 (submitted)
    • 72. User survey (broadcast professionals)
    • 73. Sound and Vision: Education
      • Government and ‘Images for the Future’
      • 74. Earlier Initiatives
      • 75. ED*IT latest development completed with tools for
      teacher and student
      • ED*IT has been tested and developed in cooperation with many schools
    • ED*IT: Proposition
      • One environment provides access to different
      controlled content databases (video, audio, photograps, articles, etc)
      • Editorial Staff contextualizes and enriches
      content for educational use
      • Enriched with tools for student and teacher to edit
      content in an easy way
      • For primary-, secondary- and vocational education
    • ED*IT: Functionalities
      Cut Videoclips
      Digital Paper Maker
      Presentation Maker
      Edit Photographs
      Video & Content Editor
      Upload Files
      Dossier Maker
      Teacher Forum
      E- Lesson Maker
      Timeline Maker
    • 76. ED*IT: Facts & Figures
      • Test Accounts: 2500
      • 77. Licence: 50 schools
      • 78. Licence: 50 educational departments
      • 79. Objective: Same market share as Teleblik is 78%
    • Researchers
      • Verteld Verleden aims at establishing a shared information space on distributed Dutch Oral History collections:
      • 80. distributed collections (harvested via OAI)
      • 81. search & interlink collections via centralized search
      • 82. project goals:
      provide demonstrator portal to show how technology could help researchers
      acquire information on specific user requirements
      dedicated work space
    • 83. example VPRO radio interviews
    • 84. interaction requirements
      people expect easy interaction as
      in 'every-day tools' they use on the web ...
      • The Sound and Vision Experience: a crossover between a museum and amusement park with various archive material, to make audiovisual heritage more acessible to the general public
    • interaction
      • people expect easy interaction as the in 'every-day tools' they use on the web
      • 85. next generation interaction:
      • 86. hyperlinked video
      • 87. interactive and collaborative interaction modalities
      • 88. a truly Internet connected society
    • draw communities into libraries
    • 89. goals
      • exploiting community tagging (tagging games, etc)
      • 90. exploring the wisdom of crowds by hooking up with user communities (e.g., everyone-as-commentator, unexpected experts)
      • 91. capturing relevant information from the internet and aligning this with archived items.
      • 92. finding new ways for communities to interact with the data.
    • Technology perspective
      • provide anchor points for linking up with the `cloud’ (entity detection, segmentation, cross-collection SID, etc): people, places, events, topics, quotes, etc.
      • 93. keywords: reliability, speed
      • 94. synchronization of UGC with AV documents
      • 95. users in the loop: UGC for adapting/training analysis tools
      • 96. early fushion of multiple modalities (vision, speech)
      • 97. technology aided annotation: Documentalist Support System
    • 98. Crowdsourcing
      14 minutes left for annotation
      you score when somebody else uses the same term
      fill in words that describe what you see or hear
    • 99. Tagging game example
    • 100. Play against ASR
    • 101.
    • 102. Hollands Glorie op Pinkpop
    • 103. Wrap up
      • value of archive is strongly related to access opportunities
      • 104. access is to a large extend technology driven
      • 105. but next to technology development we need to make a shift:
      • 106. from a ‘laboratory view’ on users to drawing users and communities into the loop
      • 107. NISV is aiming towards this two-way strategy:
      • 108. incorporate advanced access technology
      • 109. discuss access requirements with the stakeholders