The document summarizes information about the MediaEval 2014 Multimedia Benchmark Workshop. It provides details about the workshop location in Barcelona, tasks being presented, history and organization of MediaEval workshops, and thank yous to sponsors and organizers. Participants are welcomed and encouraged to present their work, discuss solutions, and plan future collaborations. Information is also provided about technical retreats, poster sessions, surveys, papers, and proposing new tasks for 2015.

1. MediaEval 2014
Multimedia Benchmark Workshop
16-17 October 2014
Reial Acadèmia de Bones Lletres de Barcelona

2. Welcome to MediaEval
• We have arrived at the culmination of 2014.
• We gather to:
– Present our algorithms,
– Discuss our solutions,
– Lay pathways for future work.
Read more about the tasks in the program notes
of the MediaEval 2014 Workshop.

3. The MediaEval Benchmark…
• …evaluates new algorithms for multimedia
access and retrieval.
• …emphasizes the "multi" in multimedia:
speech, audio, visual content, tags, users,
context.
• …innovates new tasks and techniques focusing
on the human and social aspects of
multimedia content.

4. MediaEval History
• MediaEval was founded as VideoCLEF in 2008.
• Ran in 2008 and 2009 at the Cross Language
Evaluation Forum (CLEF) and became MediaEval in
2010.
• Ran in 2010 and 2011 as an independent benchmark
under the sponsorship of the PetaMedia Network of
Excellence.
• In 2012, ran as a completely “bottom-up benchmark”
with no parent project.
• In 2014, workshop was not co-located with a major
conference.

5. The 11 MediaEval 2014 Tasks
• (New!) C@MERATA: Question Answering on Classical Music Scores
• (New!) Crowdsourcing: Crowdsorting Timed Comments about Music
• Diverse images: Retrieving Diverse Social Images
• Emotion in Music
• Placing: Multimodal Location Estimation
• QUESST: Query by Example Search on Speech (ex SWS) Task
• Search and Hyperlinking
• SED: Social Event Detection in Web Multimedia
• (New!) SEM: Synchronization of multi-user Event Media
• Visual Privacy
• Violence: Violent Scenes Detection

6. 2014 Survey Responses
0
50
100
150
200
2011 2012 2013 2014

7. 2014 Working Notes Papers
In 2014, we had 11 task overview papers (grey) and 85 accepted participant papers (black).

8. 2014 Workshop Participants

9. Poster Session
• Each poster session has a focus task or tasks,
marked in the program.
• Next is the “ThursAM” session, dedicated to
Social Event Detection.
• QUESST people can start putting up their
posters around the perimeter.
• During lunch we will have the “QUESST
quest”: guided tour of QUESST posters.

10. Internet
• We follow the grand tradition of Santa Croce in
Fossabanda.
• Wireless internet is provided solely for three
purposes:
1. turning on your
out-of-office auto-reply
2. the occasional tweet
#mediaeval14
3. For Task organizers
to submit MDMs

11. Technical Retreats
• Purpose
– Detailed technical discussion of tasks
– Making future plans
– All are welcome, but mostly interesting for task
participants
• Place
– Your task organizers will tell you where the technical
retreats are.
– Task organizers please talk to the Technical Retreat
chair, to know where your retreat is.

12. Workshop Keynote
• MediaEval originally arose in CLEF (a European
information retrieval benchmark)
• We have a tradition of networking with other
benchmarks.
• This includes a keynote speaker from another
benchmark each year.
• This year: Sergio Escalera from Universitat de
Barcelona “ChaLearn Challenge and
Workshop.

13. Slides
• Please load your slides on this computer in the
break just before your session.
• There is a folder for your task on the desktop.
• If you do not want your slides online, please
delete them from the folder after the session.

14. Bibliography
• MediaEval Group
• http://www.citeulike.org/groupfunc/16499/h
ome
• Please post your publications that use
MediaEval data to the bibliography.
• Important for understanding the contribution
of MediaEval tasks to the research community.

16. Resources
• Tasks publish their own data sets.
• Please send me a link and I will add it to the
website:
http://www.multimediaeval.org/datasets
• Many tasks submit papers to the MMSys Data Set
trackhttp://www.mmsys.org/index.php/2015-
mmsys-data-track
(Dataset paper Submission Deadline: November
10, 2014)

17. Final Session
• MDMs are “MediaEval Distinctive Mentions”
• Task organizers assign MDMs to point out an
interesting scientific contribution (that is not
necessarily top ranked contribution).
• Task organizers also assign MDMs to praise or
tease participants or each others.
• The final session of the workshop is devoted
to MDMs and outlook on next year.

18. MediaEval Letters
• “MediaEval Letters” is a section in the working
notes papers that will contain papers on topics
other than tasks.
• Examples:
– Discussions of reproducibility and reproduction
– Documentation of best practices
– Multi-year retrospectives on tasks
• Scheduled to be introduced in 2015.

19. Thank you’s

20. Workshop Organization
Local Chair
Xavier Anguera (Telefonica Research, Spain)
Coordination Chair
Mohammad Soleymani (University of Geneva, Switzerland)
Poster Chairs
Bogdan Ionescu (University Politehnica of Bucharest, Romania)
Xavier Giró (Universitat Politecnica de Catalunya, Barcelona)
Logistics Chair
Saskia Peters (Delft University of Technology, Netherlands)

21. Workshop Organization
Filmmaker
Olivier Van Laere and company
Technical Retreat Chair
Michele Trevisiol
Proceedings Chair
Michael Riegler, Simula Research Lab AS, Norway
Documentation Chair
Richard Sutcliffe, University of Essex, UK

22. Workshop Organization
Resources and Recognition committee (MediaEval Community Council)
Mohammad Soleymani (University of Geneva, Switzerland)
Guillaume Gravier (IRISA, France)
Bogdan Ionescu (University Politehnica of Bucharest, Romania)
Gareth Jones (Dublin City University, Ireland)
Martha Larson (Delft University of Technology, Netherlands)
Workshop Assistants:
Sandra Avila, Bogdan Boteanu, Shu Chen, Justin Chiu, Tom Collins, Taufik Edy
Sutanto, Maria Eskevich, Vaiva Imbrasaite, Irene Gris, Santosh Kesiraju, Daniel
Manchon, Amaia Salvador, Emanuele Sansone, Patrick Schwab, Carles Ventura.
Statistics and Research Assistants:
Bogdan Boteanu and Anca Radu (University Politehnica of Bucharest, Romania)

23. Thank you to Task Organizers
• Anna Aljanaki, Utrecht University, Netherlands
• Ahmed Al-Obaidi, University of Reading, U
• Robin Aly, University of Twente, Netherland
• Xavier Anguera, Telefonica Research, Spain
• Atta Badii, University of Reading, UK
• Andi Buzo, University Politehnica of Bucharest,
Romania
• Liangliang Cao, IBM TJ Watson, USA
• Shu Chen, Dublin City University, Ireland
• Jaeyoung Choi, ICSI, USA

24. Thank you to Task Organizers
• Nicola Conci, University of Trento, Italy
• Tim Crawford, Goldsmiths College, University of
London, UK
• Francesco De Natale, University of Trento, Italy
• Touradj Ebrahimi, EPFL, Switzerland
• Volker Eiselein, Technische Universität Berlin, German
• Maria Eskevich, Dublin City University, Irelan
• Christian Fedorczak, Thales Security Solutions &
Services
• Gerald Friedland, ICSI, US

25. Thank you to Task Organizers
• Chris Fox, University of Essex, UK
• Eduard Hovy, Carnegie Mellon University, US
• Bogdan Ionescu, University Politehnica of Bucharest,
Romania
• Yu-Gang Jiang, Fudan University, Shanghai, China
• Gareth Jones, Dublin City University, Ireland
• Pavel Korshunov, EPFL, Switzerland
• Martha Larson, Delft University of Technology,
Netherlands
• Mihai Lupu, Vienna University of Technology, Austria

26. Thank you to Task Organizers
• Vu Lam Quang, University of Information Technology, VNU-
HCMC, Vietnam
• David Racca, Dublin City University, Ireland
• Giuseppe Rizzo, Univ. of Torino, Italy
• Luis Javier Rodriguez-Fuentes, University of the Basque
Country, Spain
• Deane L. Root, University of Pittsburgh, US
• Markus Schedl, Johannes Kepler University, Linz, Austria
• Mats Sjöberg, University of Helsinki, Finland
• Mohammad Soleymani, University of Geneva, Switzerland

27. Thank you to Task Organizers
• Richard Sutcliffe, University of Essex, UK
• Igor Szőke, Brno University of Technology ,
Czech Republic
• Bart Thomee, Yahoo Labs, San Francisco, USA
• Raphael Troncy, EURECOM, France
• Karthik Yadati, Delft University of Technology,
Netherlands
• Yi-Hsuan Yang, Academia Sinica, Taiwan

28. MediaEval Supporters
• ISCA SIG Speech and Language in Multimedia
(contact at workshop: Martha or Gareth)
• IEEE STC Social Networking
• IEEE SIG Audio and Speech Processing for
Multimedia (contact at workshop: Xavi
Anguera)

29. Thank you to sponsors
• ELIAS ESF Research Networking Programme on
Evaluating Information Access Systems.
• Friends of SIGIR.
• Technicolor.
• COMMIT/ Dutch national project.

30. Thank you
• Thank you to the:
Reial Acadèmia de
Bones Lletres de Barcelona
• Thank you to UPF for
the piano.

31. How to propose a 2015 task?
• Come to the meeting tomorrow 10-13 here.
• Send in a task proposal:
http://www.multimediaeval.org/mediaeval2015
• Proposal deadline: 22 December 2014
• Tasks are decided by amount of interest shown
on the survey.
• Main tasks must have five participants who vow
to cross the finish line come hell or high water.