1. The LEMO Annotation Framework
Open Annotations Meeting
Berkeley, 10/12/2009
Bernhard Haslhofer
Bernhard Haslhofer

2. What is LEMO?
• A Web-based, multimedia annotation framework,
consisting of
• a back-end storage
• front-ends for image, audio, and video annotation
• Demo at: http://dme.arcs.ac.at/annotation/econnect-
annotation-showcase.html
Bernhard Haslhofer 2

4. Driving Requirements...
1. objects to be annotated are multimedia
‣ need a uniform model; no content-type specific solution
2. annotations address specific content parts (fragments)
‣ need a uniform fragment identification approach
3. shift towards the Web
‣ need to expose annotation data on the Web (Linked Data)
Bernhard Haslhofer 4

5. Reuse of existing standards
• Semantic Web standards (RDF/S, OWL, SPARQL)
• RESTful Web Service Design
• Annotea data model & protocol
• XPointer, MPEG-21, SVG, etc. for fragment
identification
• ....
Bernhard Haslhofer 5

6. Why not use Annotea as it is?
• Annotea is for the annotation of Web pages; LEMO
should also support annotation of embedded content
items
• Annotea doesn’t assign datatypes to its properties;
LEMO uses OWL’s (data)typing features
• Annotea doesn’t support multimedia fragment
identification; LEMO uses MPEG-21 (multimedia)
fragment identification
Bernhard Haslhofer 6

7. A Uniform Annotation Model
• annotation profiles; cf. application profiles for metadata
interoperability
• an Annotea-based core model
• with content- and annotation-type specific model
extensions
• implemented as light-weight software components
(add-ons)
Bernhard Haslhofer 7

8. A Uniform Annotation Model
Annotea Annotation Schema: http://www.w3.org/2000/10/annotation-ns#
Annotation
annotates author body created modified context related
subClassOf subPropertyOf subPropertyOf subPropertyOf subPropertyOf subPropertyOf subPropertyOf
annotates author label created modified fragment
date date any
string string
Time Time URI
Annotation
Annotation Core Schema: http://lemo.mminf.univie.ac.at/annotation-core#
Key Symbols:
RDFS/OWL RDFS OWL OWL XML Schema
domain range
Class Property ObjectProperty DatatypeProperty Datatype
Bernhard Haslhofer 8

9. A Uniform Annotation Model
Annotation Relationship Schema: Text Annotation Schema:
http://lemo.mminf.univie.ac.at/ann-relationship# http://lemo.mminf.univie.ac.at/ann-text#
Relation- Text
ship Annotation
isLinkedTo title description
subClassOf subCla
ssOf
subPropertyOf subPropertyOf
annotates author label created modified fragment
date date any
string string
Time Time URI
Annotation
Annotation Core Schema: http://lemo.mminf.univie.ac.at/annotation-core#
Bernhard Haslhofer 9

10. Uniform Fragment Identification
• introduce a fragment element with URI range
• follow MPEG-21 media pointer scheme
• an annotation can define multiple fragments
• URI-encoding of fragments
• minimal consensus
• content-type specific extensions
Bernhard Haslhofer 10

11. Uniform Fragment Identification
12
< rdf : RDF Fragment
xmlns : a =" http :// lemo . mminf . univie . ac . at / cation can int
annotation - core #"
..." > for all approa
ized and wide
...
useful to reta
<a : annotates > http :// www . univie . ac . at / test . mpg identifier can
</ a : annotates > fragment part
<a : fragment >
http :// www . univie . ac . at / test . mpg # mp (~ time requested res
( ’ npt ’ , ’30 ’ , ’40 ’) ) losing the exa
</ a : fragment >
relationship t
... By using
</ rdf : RDF >
improving the
representation
Listing 1 Time fragment of a video expressed according to lieve, as Geurt
the MPEG-21 fragment identification specification tous use of UR
interoperable,
Bernhard Haslhofer 11 annotations.

12. <a : annotates > http :// www . univie . ac . at / test . mpg identifier can not be processed by a
</ a : annotates > fragment part of the respective UR
<a : fragment >
http :// www . univie . ac . at / test . mpg # mp (~ time requested resource is returned. Wi
( ’ npt ’ , ’30 ’ , ’40 ’) ) losing the exact fragment, this beh
</ a : fragment >
relationship to the resource as a wh
... By using this simple method in
improving the interoperability of fra
Uniform Fragment Identification
</ rdf : RDF >
representations in diverse annotatio
Listing 1 Time fragment of a video expressed according to lieve, as Geurts et al [20] have conclu
the MPEG-21 fragment identification specification tous use of URIs will help to solve the
interoperable, explicit links between
annotations.
< rdf : RDF
xmlns : a =" http :// lemo . mminf . univie . ac . at /
annotation - core #"
xmlns : x =" http :// lemo . mminf . univie . ac . at / 4.4 Exposing Annotations as Web R
annotation - video #"
..." >
Since we follow a REST-based appro
... URIs (e.g., http://www.example.o
are in fact dereferencable URIs, whi
<a : annotates > http :// www . univie . ac . at / test . mpg
</ a : annotates > by humans and machines. Therefor
<a : fragment > tation framework fulfills the first and
http :// www . univie . ac . at / test . mpg # mp (~ time
( ’ npt ’ , ’30 ’ , ’40 ’) ) principles, as described in Section 3
</ a : fragment > To fulfill the third principle, LEM
<x : time_fragment xmlns : mpeg21 =" http :// lemo .
mminf . univie . ac . at / an not ati on_m peg 21 #" >
expose annotation data in different
< mpeg21 : uri_fid > http :// www . univie . ac . at / Humans typically access Web resour
test . mpg # mp (~ time ( ’ npt ’ , ’30 ’ , ’40 ’) ) which in turn requires an (X)HTM
</ mpeg21 : uri_fid >
< mpeg21 : time_scheme > npt </ mpeg21 : order to display the returned inform
time_scheme > requirement by relying on content n
< mpeg21 : start_time >30 </ mpeg21 : start_time >
< mpeg21 : end_time >40 </ mpeg21 : end_time > a built-in HTTP feature. Figure 5 il
</ x : time_fragment > tations can be retrieved in various fo
...
the appropriate mime-type in the H
header field. LEMO forwards client r
</ rdf : RDF > annotation (e.g., http://example.c
Listing 2 Alternative representation within the add-on to the appropriate physical represen
model example.com/annotations/html/1
and http://example.com/annotati
Bernhard Haslhofer 12 ing an HTTP 303 See Other respon

13. Annotations as Web Resources
Resource
1
+ URI
annotates
0..* Annotation
Bernhard Haslhofer 13

14. Annotations as Web Resources
Dereference an annotation URI, requesting HTML content
GET http://example.com/annotations/1
Accept: text/html
303 See Other
Location http://example.com/annotations/html/1
GET http://example.com/annotations/html/1
Client LEMO
Accept: text/html
200 OK
<HTML>
...
</HTML>
Dereference an annotation URI, requesting RDF content
GET http://example.com/annotations/1
Accept: application/rdf+xml
303 See Other
Location http://example.com/annotations/rdf/1
GET http://example.com/annotations/rdf/1
Client LEMO
Accept: application/rdf+xml
200 OK
<RDF>
...
</RDF>
Bernhard Haslhofer 14

15. Current LEMO developments
• integration into Europeana portal
• enhancing multimedia support on user-interface
• integration with semi-automatic (multimedia)
metadata extraction techniques
• investigating adoption of fragment identification
working groups recommendations
Bernhard Haslhofer 15