• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
CONTENTUS—A Semantic Multimodal Search User Interface
 

CONTENTUS—A Semantic Multimodal Search User Interface

on

  • 1,223 views

Is the main challenge in contemporary information retrieval and semantic search to find the needle ...

Is the main challenge in contemporary information retrieval and semantic search to find the needle
in the haystack? This presentation deals with new issues in this context and demos CONTENTUS, a modern appriach to semantic information retrieval.

There are the slides of the demo at the Visual Interfaces to the Social and Semantic Web Workshop (VISSW 2011) held in conjunction with the International Conference on Intelligent User Interfaces (IUI 2011) on February 13, 2011.

Statistics

Views

Total Views
1,223
Views on SlideShare
1,190
Embed Views
33

Actions

Likes
1
Downloads
12
Comments
0

2 Embeds 33

http://www.johannes-p-osterhoff.com 31
http://www.goread.io 2

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    CONTENTUS—A Semantic Multimodal Search User Interface CONTENTUS—A Semantic Multimodal Search User Interface Presentation Transcript

    • More than the Sum of its PartsCONTENTUS—A Semantic MultimodalSearch User InterfaceJörg Waitelonis, Johannes Osterhoff, Harald SackVisual Interfaces to the Social and Semantic Web (VISSW 2011)In conjunction with the International Conference on IntelligentUser Interfaces (IUI 2011), J. Osterhoff, February 13, 2011
    • CONTENTUS? A Teaser2 Semantic Search Engine Cultural Heritage Domain Media types Articles, Books, Video, Audio, … People, Locations, … Part of the HPI research project Semantic Media Explorer VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • CONTENTUS? A Teaser3 VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 4 Issue #1 How to find the needle in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 5Image: iStockphoto VISSW 2011 | J. Osterhoff | February 13, 2011
    • 6 needle VISSW 2011 | J. Osterhoff | February 13, 2011
    • Google Results7 VISSW 2011 | J. Osterhoff | February 13, 2011
    • Needless to say…8 needless VISSW 2011 | J. Osterhoff | February 13, 2011
    • 9 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
    • 10 Jimmy Needles Image: Jay P. Lee American basketball coach 1900—1969 Person VISSW 2011 | J. Osterhoff | February 13, 2011
    • 11 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
    • 12 Needles, CA Location Image: Eddie Biggs VISSW 2011 | J. Osterhoff | February 13, 2011
    • 13 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
    • 14 “Needle” Blockbuster by John V. Soto from 2010 Film Image: IMDB VISSW 2011 | J. Osterhoff | February 13, 2011
    • 15 Issue #1 How to find different “needles” in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 16 Issue #2 How to present “needles” as search results? VISSW 2011 | J. Osterhoff | February 13, 2011
    • Google Results17 VISSW 2011 | J. Osterhoff | February 13, 2011
    • Flickr Results18 VISSW 2011 | J. Osterhoff | February 13, 2011
    • YouTube Results19 VISSW 2011 | J. Osterhoff | February 13, 2011
    • Wikipedia Results20 VISSW 2011 | J. Osterhoff | February 13, 2011
    • 21 Issue #2 How to present different “needles” as distinguishable search results on one page? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 22 Why? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 23 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
    • 24 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
    • Image: Library of Congress25 Books VISSW 2011 | J. Osterhoff | February 13, 2011
    • 26 Articles VISSW 2011 | J. Osterhoff | February 13, 2011
    • Image: Library of Congress27 Video VISSW 2011 | J. Osterhoff | February 13, 2011
    • 28 Audio VISSW 2011 | J. Osterhoff | February 13, 2011
    • 29 Issue #3 How to make the German cultural heritage available? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 30 Authors i.e. Bertolt Brecht VISSW 2011 | J. Osterhoff | February 13, 2011
    • 31 Years i.e. 1848 VISSW 2011 | J. Osterhoff | February 13, 2011
    • Peoplei.e. from Ulm
    • Image: Wikipedia (Softeis)33 Locations among many others… VISSW 2011 | J. Osterhoff | February 13, 2011
    • Image: Wikipedia34 Music Style VISSW 2011 | J. Osterhoff | February 13, 2011
    • Process Chain35 HPI Research VISSW 2011 | J. Osterhoff | February 13, 2011
    • On the Interface36 Media Assets Books, Articles, Video, Audio Cultural Categorizations Authors, Year of Creation, People, Locations, Music Styles, Organizations etc. VISSW 2011 | J. Osterhoff | February 13, 2011
    • 37 Issue #3 How to make the German cultural heritage available in meaningful information retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
    • 38 VISSW 2011 | J. Osterhoff | February 13, 2011
    • 39 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction40 ■ a semantic search engine and a multimedia retrieval system VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction41 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction42 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction43 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search ■ developed in the context of the German research project THESEUS VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction44 ■ Combination of a facetted browsing and breadcrumb-based navigation ■ A time-line enables time-based filtering of search results ■ Personalized search (in the next demonstrator) VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction45 “More than the sum of its parts” VISSW 2011 | J. Osterhoff | February 13, 2011
    • What is CONTENTUS? Introduction46 More efficient and more pleasant user experience. VISSW 2011 | J. Osterhoff | February 13, 2011
    • What is CONTENTUS? Context47 ■ new technologies for managing, organizing, and searching multimedia content gets more efficient ■ tremendous diversity of multimedia data affects the retrieval and search process ■ image and video search technology is evolving rapidly VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • What is CONTENTUS? Context48 ■ semantic search technology promises more accurate and complete search results ■ new user interfaces are required to enable query formulation and intelligent visualization of search results VISSW 2011 | J. Osterhoff | February 13, 2011
    • What is CONTENTUS? Technologies49 ■ all types of media assets ■ technologies  media optimization  content analysis  semantic metadata generation  speech transcripts  OCR  extraction of semantic entities (i.e. people, places, events) ■ highly heterogeneous collection of semantically annotated data VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 50 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
    • User Interface Paradigms51 No research on a single aspect of current approaches in semantic multimedia retrieval… VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms52 But: exploiting synergies by combining state-of-the-art technologies instead. VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms53 semantic multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms54 semantic + multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms55 semantic + multimedia technologies + interface design VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms56 combination of multimedia and semantic search = new challenges in visualizing search results and facilitating navigation VISSW 2011 | J. Osterhoff | February 13, 2011
    • User Interface Paradigms57  compare different media at one glance  how to summarize content most meaningfully in one single search result item  time-based information (audio, video) VISSW 2011 | J. Osterhoff | February 13, 2011
    • User Interface Paradigms58 multimodal VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms59 multimodal = articles, books, video, audio, people, locations on one result-page VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms60 The route with the user‟s experience gets more important than the original goal of fact finding. VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Paradigms61 main goal satisfying search experience for the user VISSW 2011 | J. Osterhoff | February 13, 2011
    • 62
    • 63 Search Area
    • 64 Search Field breadcrumbs
    • 65
    • 66
    • 67 Result Area
    • 68 media types history collection organize and filter
    • 69 time-line faceted suggestions refine and filter
    • search results70
    • search results71 article article video audio location person video
    • 72
    • 73
    • 74
    • 75
    • 76 less important search results
    • 77
    • User Interface Detail Page for Books78 Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • User Interface Detail Page for Books79 full page other pages articles on this page text snippet (OCR) detected entities Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 80 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
    • 81 Demo http://mediaglobe.yovisto.com:8081/contentus/ VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 82 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
    • Summary83 Issue #1 Issue #1 How to find the needle How to find different in the haystack? “needles” in the haystack? Issue #2 Issue #2 How to present How to present “needles” different “needles” as as search results? distinguishable search results? Issue #3 Issue #3 How to make the German How to make the German cultural heritage available? Cultural Heritage available in meaningful Information Retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
    • Future Work84 ■ Feedback that lets the user know why exactly the listed items have been presented ■ Possibility to connect with friends and to share results with them ■ Suggestions based on friends„ likes ■ State-of-the-art multi-platform web design ■ Next demonstrator to be presented at IBC 2011 Amsterdam VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 85 VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • Person Book86 Location Audio Video VISSW 2011 | Johannes Osterhoff | February 13, 2011
    • 87 Thank you! Johannes P Osterhoff, M.A. johannes.osterhoff@hpi.uni-potsdam.de Hasso-Plattner-Institute, Prof.-Dr.-Helmert-Str. 2-3, 14482 Potsdam, Germany VISSW 2011 | Johannes Osterhoff | February 13, 2011