Institute of Information Systems & Information Management RDFa-deployed Multimedia Metadata Werner Bailer & Michael Hausenblas Graz, 2007-08-10

Where? Use Cases What constitutes a ramm.x use case? Media assets are published on the Web Along with the media asset, the multimedia metadata (as MPEG-7, Exif, ID3, etc.) is published The metadata itself are neither free-text nor natively represented using an RDF-based vocabulary (as Music Ontology or OntologyX3D ) A Semantic Web agent wants to access the multimedia metadata

What constitutes a ramm.x use case?

Media assets are published on the Web

Along with the media asset, the multimedia metadata (as MPEG-7, Exif, ID3, etc.) is published

The metadata itself are neither free-text nor natively represented using an RDF-based vocabulary (as Music Ontology or OntologyX3D )

A Semantic Web agent wants to access the multimedia metadata

Use Case: Photo Sharing (1) Exif information contains a lot of useful metadata Some photo sharing sites (as Flickr and PBase ) extract & display metadata

Exif information contains a lot of useful metadata

Some photo sharing sites (as Flickr and PBase ) extract & display metadata

Uses Case: Photo Sharing (2) Exif metadata in image is standardized But how to get it from a photo sharing site? One can scrap the HTML code Use a web service, e.g. flickr.photos.getExif , returns proprietary XML: <photo id=&quot;4424&quot; secret=&quot;06b8e43bc7&quot; server=&quot;2&quot;> <exif tagspace=&quot;TIFF&quot; tagspaceid=&quot;1&quot; tag=&quot;271&quot; label=&quot;Manufacturer&quot;> <raw>Canon</raw> </exif> <exif tagspace=&quot;GPS&quot; tagspaceid=&quot;3&quot; tag=&quot;4&quot; label=&quot;Longitude&quot;> <raw>64/1, 42/1, 4414/100</raw> <clean>64° 42' 44.14&quot;</clean> </exif> </photo> but no interoperability between different services!

Exif metadata in image is standardized

But how to get it from a photo sharing site?

One can scrap the HTML code

Use a web service, e.g. flickr.photos.getExif , returns proprietary XML:

<photo id=&quot;4424&quot; secret=&quot;06b8e43bc7&quot; server=&quot;2&quot;>

<exif tagspace=&quot;TIFF&quot; tagspaceid=&quot;1&quot; tag=&quot;271&quot; label=&quot;Manufacturer&quot;>

<raw>Canon</raw>

</exif>

<exif tagspace=&quot;GPS&quot; tagspaceid=&quot;3&quot; tag=&quot;4&quot; label=&quot;Longitude&quot;>

<raw>64/1, 42/1, 4414/100</raw>

<clean>64° 42' 44.14&quot;</clean>

</exif>

</photo>

but no interoperability between different services!

Use Case: Buy Music (1) Online music stores provide little metadata (e.g. iTunes Store )

Online music stores provide little metadata (e.g. iTunes Store )

Use Case: Buy Music (2) M etadata are available in common formats, as for example ID3 Comprehensive online music databases (e.g. http://musicbrainz.org/ ) exist, providing metadata either as proprietary XML or RDF Offering music with metadata allows linking with Semantic Web resources about artists containing reviews recommendations etc.

M etadata are available in common formats, as for example ID3

Comprehensive online music databases (e.g. http://musicbrainz.org/ ) exist, providing metadata either as proprietary XML or RDF

Offering music with metadata allows linking with Semantic Web resources

about artists

containing reviews

recommendations

etc.

Use Case: Describe Video Structure NBA publishes game summaries on YouTube Published metadata are very poor As content is edited, information about the structure of the clips is available, but YouTube does not support it Video structure could for example be published as MPEG-7 description allows accessing parts of the video, as scenes showing a certain player There may be a LOT of metadata, hence a kind of ”streaming” would be useful

NBA publishes game summaries on YouTube

Published metadata are very poor

As content is edited, information about the structure of the clips is available, but YouTube does not support it

Video structure

could for example be published as MPEG-7 description

allows accessing parts of the video, as scenes showing a certain player

There may be a LOT of metadata, hence a kind of ”streaming” would be useful

Use Case: Publish Professional Content Content providers sell their content increasingly online, for example the BBC Motion Gallery Metadata published with content are not accessible for auto- matic processing Comprehensive metadata are available in-house in established representations (as SMEF, or EBU P_Meta)

Content providers sell their content increasingly online, for example the BBC Motion Gallery

Metadata published with content are not accessible for auto- matic processing

Comprehensive metadata are available in-house in established representations (as SMEF, or EBU P_Meta)

Use Case: Rights Metadata In simple cases, rights metadata are just a link to a well-defined license (as Creative Commons ) More complicated rights information are usually available as huge text-only documents (e.g. Getty Images site) DRM metadata formats are available, e.g. in MPEG-21, a REL is defined

In simple cases, rights metadata are just a link to a well-defined license (as Creative Commons )

More complicated rights information are usually available as huge text-only documents (e.g. Getty Images site)

DRM metadata formats are available, e.g. in MPEG-21, a REL is defined

Requirements Embed references to media assets in existing multimedia metadata formats in (X)HTML Describe not only complete (X)HTML pages, but also certain parts of it Resulting description must be usable by an Semantic Web agent Provide reference to services capable of mapping between a specific multimedia metadata format and RDF (formalisation) Several descriptions may be available for a media asset (e.g. in different formats, covering different aspects) and there may be several ways to formalise the description available in a certain format Due to amount of metadata, it may be necessary to &quot;stream&quot; the metadata in pieces related to spatio-temporal segments

Embed references to media assets in existing multimedia metadata formats in (X)HTML

Describe not only complete (X)HTML pages, but also certain parts of it

Resulting description must be usable by an Semantic Web agent

Provide reference to services capable of mapping between a specific multimedia metadata format and RDF (formalisation)

Several descriptions may be available for a media asset (e.g. in different formats, covering different aspects) and there may be several ways to formalise the description available in a certain format

Due to amount of metadata, it may be necessary to &quot;stream&quot; the metadata in pieces related to spatio-temporal segments

Institute of Information Systems & Information Management RDFa-deployed Multimedia Metadata Werner Bailer & Michael Hausenblas Graz, 2007-08-10