Talk at the Building Web Observatories workshop at WebSci'13. Paris, France. 1st May 2013. http://eprints.soton.ac.uk/352465/ The data contained within the web is inherently multimedia; consisting of a rich mix of textual, visual and audio modalities. Prospective Web Observatories need to take this into account from the ground up. This paper explores some uses for the automatic analysis of multimedia data within a Web Observatory, and describes a potential platform for an extensible and scalable multimedia Web Observatory.

1. BUILDING A MULTIMEDIA WEB
OBSERVATORY PLATFORM
Jonathon Hare
jsh2@ecs.soton.ac.uk

2. WHY MULTIMEDIA?

3. THE WEB IS MULTIMEDIA

4. MULTIMEDIA GIVES US A
MORE COMPLETE PICTURE OF
THE WEB, BUT ALSO A MORE
COMPLETE PICTURE OF
SOCIETY.

5. HOW CAN WE EXPLOIT
MULTIMEDIA DATA?

6. INTERLINKING CONTENT
THROUGH MM FEATURES
• Detection of near-duplicates using
MM analysis can interlink content
across languages, domains and
groups.
• Aggregating documents
about the same subject/event/
opinion
• Finding cases where media is
used in differing contexts is
also interesting.
• Exploring how different social
groups talk about the same
media

7. • Detection & recognition of people
• General object, logo, etc. detection & recognition
• Detection of locations
• Temporal detection of duplicates
MINING ENTITIES,TRENDS,
TOPICS AND EVENTS

8. • Facial analysis of visual data
• Course-grained automatic classification
(sentiment/privacy/attractiveness)
• Investigating Correlations between images and
opinions mined from text
UNDERSTANDING
MEANING OF
CONTENT

9. THE ARCOMEM APPROACH

10. •Intelligently harvest data from the web and social web
around specific ETOEs.
•Provide a scalable and extensible platform for analysis
taking into account all modalities of data.
•Expose the results of the analysis in the form of semantic
knowledge.
•Export the data in standardised formats for preservation
and exchange.