E-Librarian Service

Natural language search in multimedia knowledge bases
Serge Linckels & Christoph Meinel
Hasso Plattner Institute
at Potsdam University

Hasso Plattner Institute
2
Founded in 1998
10 professors
100 lectures, co-workers…
IT-Systems Engineering (Bachelor, Master, PhD)

Classical approach
4
strong AI 
Logical reasoning
Knowledge representation
Natural Language Processing
Machine Learning
…
weak AI 
artifical intelligence (AI)
art or porn?

Web 2.0 approach
5
voting
tagging
art or porn?
photography
woman
black & white
18-200mm VR
best picture 2010
Statistical solution
Requires critical mass of "good" users

Semantic Web approach
6
art or porn?
artistic black and
white picture
of a nakedwoman
in a narrowstreet

Keyword matching
7
user query
black
and
white
of
a
woman
in
picture
naked
narrow
street
photo
nude
outdoors daylight
search engine
data
metadata
+
artistic black and
white picture
of a nakedwoman
in a narrowstreet

Keyword search
8
too much information
false interpretation
false positive

Keyword matching problems
9
black and white photo of a nude woman outdoors in daylight
of black woman in a white daylight photo and nude outdoors
does order matters?
black and white photo of a nude woman outdoors indaylight
does size matters?

Syntax tree
10
Informal language
a date
Brown tag set
noun phrase (NP), prepositional phrase (PP), adjective phrase (ADJP), verb phrase (VP)
adjective (JJ), conjuncation (CC), preposition (IN), determiner (DT), noun (NN), verb (VBZ)
21
March
2011
a date
presentation
CRP-HT

Natural language processing
11
semantic interpretation
S  Photo hasColor.BW
photoOf.(Woman isNude)
 isOutdoors.Daylight
description logics
remove “stop words”
verbs, adjectives, adverbs  roles
nouns  concepts

Martching of concept descriptions
12
artistic black and
white picture
of a naked woman
in a narrow street
P  Picture  hasColor.BW  isArtistic  pictureOf.(Woman  isNaked)  isLocated.(Street  isNarrow)
?
similarity
Q  Photo  hasColor.BW  photoOf.(Woman  isNude)

Ontological approach
13
picture
image
photo
movie
semantic resources
e.g., WordNet
pictureOf  photoOf
isNude  isNaked
equivalences
Photo  Picture
Photo is subsumed by Picture
P  Picture  hasColor.BW  isArtistic  pictureOf.(Woman  isNaked)  isLocated.(Street  isNarrow)
?
similarity
Q  Photo  Picture  hasColor.BW  pictureOf.(Woman
 isNaked)  isOutdoors.Daylight
Q  Photo  hasColor.BW  photoOf.(Woman  isNude)

Semantic distance
14
Query
Object 1
1
2
Object 2
no cover
3
Object 3
Object 4
Object 5
cover
rest
miss
Best cover = object with smallest rest and miss
preference is given to smallest miss
miss

Annotating motion picture
15
Lecture "WWW Grundlagen" by Prof. Meinel
#1
Intro
#n
How TCP/IP works
#2
Protocols in general
#3
Error-handling as task of a protocol
#4
Error-handling
LO3 Protocol hasTask.ErrorHandling
This clip is about
error-handling as
a task of a protocol
<owl:Class rdf:about="#LO3">
<owl:intersectionOf rdf:parseType="Collection">
<owl:Class rdf:about="#Protocol" />
<owl:restriction>
<owl:onProperty rdf:resource="#hasTask" />
<owl:someValuesFrom rdf:resource="#ErrorHandling" />
</owl:restriction>
</owl:intersectionOf>
</owl:Class>

Illustration
16

E-Librarian Service

by Serge Linckels on Mar 23, 2011

Accessibility

Categories

Upload Details

Usage Rights

Statistics

E-Librarian Service Presentation Transcript