Natural language search in multimedia knowledge bases<br />Serge Linckels & Christoph Meinel<br />Hasso Plattner Institute...
Hasso Plattner Institute<br />2<br />Founded in 1998<br />10 professors<br />100 lectures, co-workers…<br />IT-Systems Eng...
3<br />
Classical approach<br />4<br />strong AI    <br /><ul><li> Logical reasoning
 Knowledge representation
 Natural Language Processing
 Machine Learning
 …</li></ul>weak AI    <br />artifical intelligence (AI)<br />art or porn?<br />
Web 2.0 approach<br />5<br />voting<br />tagging<br />art or porn?<br />photography<br />woman<br />black & white<br />18-...
Semantic Web approach<br />6<br />art or porn?<br />artistic black and <br />white picture<br />of a nakedwoman<br />in a ...
Keyword matching<br />7<br />user query<br />black <br />and <br />white<br />of<br />a<br />woman <br />in<br />picture <...
Keyword search<br />8<br />too much information<br />false interpretation<br />false positive<br />
Keyword matching problems<br />9<br />black and white photo of a nude woman outdoors in daylight<br />of black woman in a ...
Syntax tree<br />10<br />Informal language<br />a date<br />Brown tag set<br />noun phrase (NP), prepositional phrase (PP)...
Natural language processing<br />11<br />semantic interpretation<br />S 	Photo hasColor.BW <br />photoOf.(Woman isN...
Martching of concept descriptions<br />12<br />artistic black and <br />white picture <br />of a naked woman <br />in a na...
Upcoming SlideShare
Loading in...5
×

E-Librarian Service

919

Published on

semantic search engine in a multimedia knowledge base with natural language input

Published in: Education, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
919
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
6
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

E-Librarian Service

  1. 1. Natural language search in multimedia knowledge bases<br />Serge Linckels & Christoph Meinel<br />Hasso Plattner Institute <br />at Potsdam University<br />
  2. 2. Hasso Plattner Institute<br />2<br />Founded in 1998<br />10 professors<br />100 lectures, co-workers…<br />IT-Systems Engineering (Bachelor, Master, PhD)<br />
  3. 3. 3<br />
  4. 4. Classical approach<br />4<br />strong AI <br /><ul><li> Logical reasoning
  5. 5. Knowledge representation
  6. 6. Natural Language Processing
  7. 7. Machine Learning
  8. 8. …</li></ul>weak AI <br />artifical intelligence (AI)<br />art or porn?<br />
  9. 9. Web 2.0 approach<br />5<br />voting<br />tagging<br />art or porn?<br />photography<br />woman<br />black & white<br />18-200mm VR<br />best picture 2010<br />Statistical solution<br />Requires critical mass of "good" users<br />
  10. 10. Semantic Web approach<br />6<br />art or porn?<br />artistic black and <br />white picture<br />of a nakedwoman<br />in a narrowstreet<br />
  11. 11. Keyword matching<br />7<br />user query<br />black <br />and <br />white<br />of<br />a<br />woman <br />in<br />picture <br />naked <br />narrow<br />street<br />photo <br />nude <br />outdoors daylight<br />search engine<br />data<br />metadata<br />+<br />artistic black and <br />white picture<br />of a nakedwoman<br />in a narrowstreet<br />
  12. 12. Keyword search<br />8<br />too much information<br />false interpretation<br />false positive<br />
  13. 13. Keyword matching problems<br />9<br />black and white photo of a nude woman outdoors in daylight<br />of black woman in a white daylight photo and nude outdoors<br />does order matters?<br />black and white photo of a nude woman outdoors indaylight<br />does size matters?<br />
  14. 14. Syntax tree<br />10<br />Informal language<br />a date<br />Brown tag set<br />noun phrase (NP), prepositional phrase (PP), adjective phrase (ADJP), verb phrase (VP)<br />adjective (JJ), conjuncation (CC), preposition (IN), determiner (DT), noun (NN), verb (VBZ)<br />21<br />March<br />2011<br />a date<br />presentation<br />CRP-HT<br />
  15. 15. Natural language processing<br />11<br />semantic interpretation<br />S  Photo hasColor.BW <br />photoOf.(Woman isNude) <br /> isOutdoors.Daylight<br />description logics<br />remove “stop words”<br />verbs, adjectives, adverbs  roles<br />nouns  concepts<br />
  16. 16. Martching of concept descriptions<br />12<br />artistic black and <br />white picture <br />of a naked woman <br />in a narrow street<br />P  Picture  hasColor.BW  isArtistic  pictureOf.(Woman  isNaked)  isLocated.(Street  isNarrow)<br />?<br />similarity<br />Q  Photo  hasColor.BW  photoOf.(Woman  isNude) <br /> isOutdoors.Daylight<br />
  17. 17. Ontological approach<br />13<br />picture<br />image<br />photo<br />movie<br />semantic resources<br />e.g., WordNet<br />pictureOf  photoOf<br />isNude  isNaked<br />equivalences<br />Photo  Picture<br />Photo is subsumed by Picture<br />P  Picture  hasColor.BW  isArtistic  pictureOf.(Woman  isNaked)  isLocated.(Street  isNarrow)<br />?<br />similarity<br />Q  Photo  Picture  hasColor.BW  pictureOf.(Woman <br /> isNaked)  isOutdoors.Daylight<br />Q  Photo  hasColor.BW  photoOf.(Woman  isNude) <br /> isOutdoors.Daylight<br />
  18. 18. Semantic distance<br />14<br />Query<br />Object 1<br />1<br />2<br />Object 2<br />no cover<br />3<br />Object 3<br />Object 4<br />Object 5<br />cover<br />rest<br />miss<br />Best cover = object with smallest rest and miss<br /> preference is given to smallest miss<br />miss<br />
  19. 19. Annotating motion picture<br />15<br />Lecture "WWW Grundlagen" by Prof. Meinel<br />#1<br />Intro<br />#n<br />How TCP/IP works<br />#2<br />Protocols in general<br />#3<br />Error-handling as task of a protocol<br />#4<br />Error-handling<br />LO3 Protocol hasTask.ErrorHandling<br />This clip is about<br />error-handling as <br />a task of a protocol<br /> <owl:Class rdf:about="#LO3"><br /> <owl:intersectionOf rdf:parseType="Collection"><br /> <owl:Class rdf:about="#Protocol" /><br /> <owl:restriction><br /> <owl:onProperty rdf:resource="#hasTask" /><br /> <owl:someValuesFrom rdf:resource="#ErrorHandling" /><br /> </owl:restriction><br /> </owl:intersectionOf><br /> </owl:Class><br />
  20. 20. Illustration<br />16<br />

×