Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Audiovisual archives and digital humanities                                       Netherlands Institute for Sound and Visi...
Netherlands Institutefor Sound and Vision
Sound and Vision R&D
Agenda                         Johan Oomen – Open archives for Digital Humanities         Roeland Ordelman         - Speec...
http://jurnsearch.wordpress.com/2013/01/13/digital-humanities-map/
Images for the Futurehttp://imagesforthefuture.com/en/news/images-future-90-seconds   @johanoomen                       *
It would take over 6 millionyears to watch the amountof video that will crossglobal IP networks eachmonth in 2016.Every se...
Known item search
Explorative searchBron M., van Gorp J., Nack F., de Rijke M., van Gorp J., de Leeuw S., "A Subjunctive Exploratory Search ...
Contextual searchhttp://zookma.science.uva.nl/linking-ui?session_id=510f98e28f034
Contextual search
Linking
Vocabularies               Over 20 million               records and growing.
Archives and DH1.  Digitisation as driver for change  •    Towards a cultural commonwealth  •    Archives as a bridge to C...
Audiovisual collections, thespoken word and user needs of  scholars in the Humanities   Observations based on related     ...
E-Research E-research• New and/or rapid ways to gain knowledge• Digital resources and information technology• Big data & d...
Emerging focus audiovisualEmerging focus on on audiovisual• Multi-modal, multi-semiotic:  • multiple layers of meaning / i...
METADATA RULES     ?
Metadata & AnnotationsMetadata & annotations• Annotations:  • General (document level)  • Specific (segment level)• Metada...
Monitoring radio transcriptsINGEST SUPERVISION // ARCHIVIST            SUPPORT:   Quickly assess quality of ASR
Spoken word search 2005-2012• Wide range of projects in various domains  • Radio    • Daily ingest: selection of programs ...
1st experiment on ASR forhumanities: access topersonal recordings of Dutchnovelist WF Hermans
Access to interviewcollection with campsurvivors World War II
Access to interview collectionsFEMINIST MOVEMENT
Alignment of transcripts for indexingINTERVIEWS ON BOMBARDEMENTOF ROTTERDAM
Access to Radio interviewsExperiments with various types of access and resultpresentation: speaker changes, speaking rate,...
Access to HistoricalSpeeches:Alignment & Linking
ACCESS TO DISTRIBUTED ORAL HISTORY COLLECTIONS•  Infrastructure for   searching collections   at various institutes in   T...
Observations on speech search• Large variation in ASR performance• Performance (and decisions on use)  should be assessed ...
Usefulness of results•  Perception of usefulness   •  Usefulness in context of search/data exploration   •  Educate / Expe...
Methodology  Methodology (1)                          (1)•  E-research is an intervention in current practices!•  Promise:...
Methodology (2)•  4 stages:   1.    Preliminary archival search         •  Browsing as a general interest         •  Purpo...
ASR for ASR for        research         research• Triple-A: Accessible, Affordable, Accurate• Individual researchers sendi...
ASR        ASR service              serviceUpload: via http, ftp, apiModel of use: •  Free test bundle (10h) •  Various sm...
Dutch QueenWilhelmina addressingthe Dutch people fromLondon during WWII
Exploring Europe’s Television Heritage inChanging Contexts Erwin Verbruggen, R&D     @erwinverb
Partner overview
Metadata                         mint.image.ece.ntua.gr/                    Based on EBUcore            Mapped to the Euro...
Euscreen PortalWWW.EUSCREEN.EU
Storylines
Collaborative design sessions    Virtual Exhibition Tool
Open access publishing with AV sourcesWWW.VIEWJOURNAL.EU
Linked Open Data PilotLOD.EUSCREEN.EU
Visualisation demosDEMO.EUSCREEN.EU
www.euscreen.eu         facebook.com/euscreen         twitter.com/euscreen2/8/13
Upcoming SlideShare
Loading in …5
×

Audiovisual archives and digital humanities

1,963 views

Published on

Contribution to the 'Opening up speech archives' conference, February 7, 2013.
By Johan Oomen, Roeland Ordelman, Erwin Verbruggen
Context: http://lukemckernan.com/2013/02/05/opening-up-speech-archives/

Published in: Technology
  • Be the first to comment

Audiovisual archives and digital humanities

  1. 1. Audiovisual archives and digital humanities Netherlands Institute for Sound and Vision Johan Oomen Head of R&D (+ researcher VU University) Roeland Ordelman Policy advisor audiovisual access (+ researcher University of Twente) Erwin Verbruggen Project manager EUscreenhttp://www.walkerart.org/calendar/2009/benches-binoculars contact: joomen@beeldengeluid.nl 8 February 2013 * #ousa2013
  2. 2. Netherlands Institutefor Sound and Vision
  3. 3. Sound and Vision R&D
  4. 4. Agenda Johan Oomen – Open archives for Digital Humanities Roeland Ordelman - Speech search and Digital Humanities Erwin Verbruggen - EUscreen and DH *
  5. 5. http://jurnsearch.wordpress.com/2013/01/13/digital-humanities-map/
  6. 6. Images for the Futurehttp://imagesforthefuture.com/en/news/images-future-90-seconds @johanoomen *
  7. 7. It would take over 6 millionyears to watch the amountof video that will crossglobal IP networks eachmonth in 2016.Every second, 1.2 millionminutes of video contentwill cross the network in2016. goal: ...be the best provider of your content http://www.cisco.com/en/US/solutions/collateral/ns341/ns525/ns537/ns705/ns827 white_paper_c11-481360_ns827_Networking_Solutions_White_Paper.htm
  8. 8. Known item search
  9. 9. Explorative searchBron M., van Gorp J., Nack F., de Rijke M., van Gorp J., de Leeuw S., "A Subjunctive Exploratory Search Interface to Support Media Studies Researchers", SIGIR 12: 35th international ACM SIGIR conference on Research and development in information retrieval,, Portland, Oregon, ACM, pp. 425-434 , August, 2012.
  10. 10. Contextual searchhttp://zookma.science.uva.nl/linking-ui?session_id=510f98e28f034
  11. 11. Contextual search
  12. 12. Linking
  13. 13. Vocabularies Over 20 million records and growing.
  14. 14. Archives and DH1.  Digitisation as driver for change •  Towards a cultural commonwealth •  Archives as a bridge to CS and DH2.  Mutual benefit •  digging into data ó adding meaning3.  From pilots to sustainable solutions •  Standards (W3C) •  In-house production system •  Shared infrastructures (i.e. CLARIAH.eu) *
  15. 15. Audiovisual collections, thespoken word and user needs of scholars in the Humanities Observations based on related work in The Netherlands 2005-2012 Roeland Ordelman @roelandordelman
  16. 16. E-Research E-research• New and/or rapid ways to gain knowledge• Digital resources and information technology• Big data & data mining (social sciences)• Digital Humanities / E-Humanities• Digitization, Infra, Tools, Standards• CLARIN.eu / DARIAH.eu
  17. 17. Emerging focus audiovisualEmerging focus on on audiovisual• Multi-modal, multi-semiotic: • multiple layers of meaning / interpretation • E.g., “quote + intonation + images + discourse”• New dimensions for scholarly research• Large investments in digitization: • Images for the Future: 200k hours of film, video and audio • Various digitization projects for scientific collections
  18. 18. METADATA RULES ?
  19. 19. Metadata & AnnotationsMetadata & annotations• Annotations: • General (document level) • Specific (segment level)• Metadata: typically sparse / document level• Requirements dependent on research field• Annotation generation: • Manual (Individual, Teams, Crowd) • Automatic: (un/lightly) supervised
  20. 20. Monitoring radio transcriptsINGEST SUPERVISION // ARCHIVIST SUPPORT: Quickly assess quality of ASR
  21. 21. Spoken word search 2005-2012• Wide range of projects in various domains • Radio • Daily ingest: selection of programs • Woord.nl: public access to radio content • Historical video collections with sparse data • ``Oral History’’• Development of an ASR service for cultural heritage institutions
  22. 22. 1st experiment on ASR forhumanities: access topersonal recordings of Dutchnovelist WF Hermans
  23. 23. Access to interviewcollection with campsurvivors World War II
  24. 24. Access to interview collectionsFEMINIST MOVEMENT
  25. 25. Alignment of transcripts for indexingINTERVIEWS ON BOMBARDEMENTOF ROTTERDAM
  26. 26. Access to Radio interviewsExperiments with various types of access and resultpresentation: speaker changes, speaking rate, searchstrategies, word clouds
  27. 27. Access to HistoricalSpeeches:Alignment & Linking
  28. 28. ACCESS TO DISTRIBUTED ORAL HISTORY COLLECTIONS•  Infrastructure for searching collections at various institutes in The Netherlands•  Harvesting of Metadata (OAI-PMH)•  ASR as a service•  Evaluated with Oral Historians
  29. 29. Observations on speech search• Large variation in ASR performance• Performance (and decisions on use) should be assessed in context of application: audiovisual search• Usefulness in audiovisual search should be assessed in context of use scenarios• Use scenarios require specific presentation/visualization requests
  30. 30. Usefulness of results•  Perception of usefulness •  Usefulness in context of search/data exploration •  Educate / Expectation management •  Guide searching •  Show why (errors, confidence, trust-levels, cut-offs) •  Focus on research needs•  Improve on ASR quality •  Educate: how to record an interview (Oral History) •  Use available textual resources (alignment, vocab optimization)•  Improve on search application •  Visualization •  Result presentation •  documents versus segments •  combination of information sources •  cross/within-collection linking
  31. 31. Methodology Methodology (1) (1)•  E-research is an intervention in current practices!•  Promise: •  increased efficiency, relevance, novelty•  Interest of scholars: • tools that facilitate or simplify existing practice (RIN report, 2011)•  Co-development ICT-researchers & scholars to adjust expectations. Examples: • Finding more in less time may not be a goal in itself for humanities researchers • Deep engagement with primary texts versus results on the segment level
  32. 32. Methodology (2)•  4 stages: 1.  Preliminary archival search •  Browsing as a general interest •  Purpose driven (checking details, complementary resources) •  Item-oriented (finding first mentioning of something) •  Collection-oriented (thematic, source, person, event) 2.  Content analysis •  Visualization, compression, aggregation •  (optionally) go back to (1) 3.  Presentation and dissemination •  Enhanced publications (persistent identifiers on segment level) 4.  Curation •  Trusted digital repository•  (spoken) search scenarios: facilitate these stages
  33. 33. ASR for ASR for research research• Triple-A: Accessible, Affordable, Accurate• Individual researchers sending files to ASR?• Embedded in suite of research tools?• What about integration in search applications? • Stagnation due to inadequate local infrastructures• Variation across collections requires ‘tailor- made’ approaches: e.g., speaker adaptation, vocabulary adaptation, alignment, collection of related resources (information trail)
  34. 34. ASR ASR service serviceUpload: via http, ftp, apiModel of use: •  Free test bundle (10h) •  Various small/medium/large bundles •  Reduced costs (only hardware and maintenance) •  Management by CH body •  Maintenance by industry partner
  35. 35. Dutch QueenWilhelmina addressingthe Dutch people fromLondon during WWII
  36. 36. Exploring Europe’s Television Heritage inChanging Contexts Erwin Verbruggen, R&D @erwinverb
  37. 37. Partner overview
  38. 38. Metadata mint.image.ece.ntua.gr/ Based on EBUcore Mapped to the Europeana Data Model MAPPING TOOL ANNOTATION TOOLMassive uploads Item and Group Level AnnotationSchema Mapping Service Connection withQuality Control EUscreen ThesauriEuropeana Preview Services Search and Browsing Services
  39. 39. Euscreen PortalWWW.EUSCREEN.EU
  40. 40. Storylines
  41. 41. Collaborative design sessions Virtual Exhibition Tool
  42. 42. Open access publishing with AV sourcesWWW.VIEWJOURNAL.EU
  43. 43. Linked Open Data PilotLOD.EUSCREEN.EU
  44. 44. Visualisation demosDEMO.EUSCREEN.EU
  45. 45. www.euscreen.eu facebook.com/euscreen twitter.com/euscreen2/8/13

×