This document summarizes the MediaDNA project, which investigates emerging fingerprinting technologies to help uncover relationships between disparate audiovisual materials. The project brings together a network of organizations to apply these technologies for public use and digital humanities research. The goals are to help search collections, understand how content circulates online, and stimulate public engagement with cultural heritage. Challenges include scaling the technologies beyond proof of concepts and addressing issues of privacy and control of online content.

1. MediaDNA
Investigating emerging technologies in Humanities
research to serve the public use of digitized
audiovisual material
DARIAH-EU Innovation Forum #1
Aarhus, November 3, 2017
Prof. Eggo Müller, Professor of Media and Communication. Utrecht University
Johan Oomen, Head of Research, Netherlands Institute for Sound and Vision

5. Video fingerprints
Three dimensions: color, shape, temporal
A “hash”:
A3a1c25d9b71a19d412188fa9ee0949a

6. Descendants view (@Fraunhofer)

7. Segment matching (@Fraunhofer)

8. MediaDNA - network
 Netherlands Institute for Sound and Vision (NL)
 Utrecht University (NL)
 Digital Humanities Lab, Umeå University (SE)
 Centre for Contemporary and Digital History (LUX)
 Institute for Contemporary European History (CZ)
 EUscreen Foundation  EUscreen.eu
 EUROCLIOH (European Association of History Educators)
 Europeana Foundation  Europeana.eu

9. MediaDNA - network

10. MediaDNA - objectives
DARIAH-EU grant theme “Public Humanities” to…
 Initiate a Trans-European network of digital humanities researchers,
audiovisual archives, educators and software developers.
 Investigate and test multimedia fingerprinting and –tracking
technologies for use in the in the public domain.
 Create understanding of how these emerging technologies can help
search collections and uncover previously hidden relations between
disparate items.
 Stimulate truly public engagement with audiovisual heritage.

11. MediaDNA - actions
 Research proposal “European History Reloaded: Circulation
and Appropriation of European audiovisual Heritage”
(CADEAH) to Horizon 2020 JPICH call “Digital Heritage” (June
2017)
 Symposium “Video Tracing and Tracking in Digital Humanities
Research” (Hilversum, 13 October 2017)
 Symposium “The Many Lives of Europe’s Audiovisual Heritage
Online” (Utrecht, 16 May 2018)
 Open access publication on the topic (2018)

12. MediaDNA - challenges
steered by tools  by questions
particular collections  the Internet
curation  machines
proof of concepts  scalability

14. Eggo Müller, Utrecht University
Johan Oomen, Netherlands Institute for Sound and Vision

15. MediaDNA - motivation
 Audiovisual material is central to our daily lives and the remembrance of
the, eventful, 20th century.
 More and more films, television and radio programmes, photographs and
related printed materials become digitized and thus are accessible online
for consumption and re-use.
 Re-use in the public sector (education, broadcasters) supports the
integration of the imaginative audiovisual to our material and collective
cultural memory, and in line with this a greater understanding of notions of
identity; of who we are and where we belong.
 At the same time though, the availability of digitized audiovisual material
fosters piracy and the 'unofficial' re-use of the resources circulating online.

16. EUscreen.eu

17. British Pathé on YouTube

18. Unidentified re-used fragment

19. Unidentified re-used fragment

20. Fingerprinting technologies
Four main application domains:
 metadata extraction and audio forensics (item-based)
 dataset analysis (segment matching, audio phylogeny)
 integrating different tools and multimodal analysis (see MICO,
OSS framework)
 privacy-aware personalization and recommendation based on
a combination of content-based recommendation and
collaborative filtering

21. Stream review(@Fraunhofer)

22. Tampering detection (@Fraunhofer)

23. Phylogeny of content(@Fraunhofer)

24. Ancestor view (@Fraunhofer)

25. European History Reloaded:
Circulation and Appropriation of
Digital European Audiovisual Heritage
Proposal to EU Horizon 2020 JPI Cultural Heritage -
Digital Heritage
 circulation an reuse of archival partner’s contributions beyond
the EUscreen platform
 tracing and tracking of reused items/fragments
 understanding popular appropriation of European heritage
 advising archives about their online policies and curation
fostering user engagement and reuse

26. MediaDNA: new research instruments
“Watermarking and fingerprinting are exiting technologies that can
facilitate critical business functions, including content identification,
copyright control, behavior tracking, copy protection and forensics. In
addition, these technologies provide the means for content owners who
embrace emerging distributions platforms, such as user generated
content websites, to develop new revenue models based on what they
previously might have considered pirated content.” (Milano 2012)

27. Symposium
“Video Tracing and Tracking in Digital
Humanities Research”
Netherlands Institute for Sound and Vision, 13 October 2017
 Explore the state of the art of fingerprinting technologies

28. Symposium
“The Many Lives of Europe’s Audiovisual
Heritage Online”
Utrecht University, 16 May 2018
 Explore the circulation and appropriation of audiovisual
heritage online