Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
More than the Sum of its PartsCONTENTUS—A Semantic MultimodalSearch User InterfaceJörg Waitelonis, Johannes Osterhoff, Har...
CONTENTUS?    A Teaser2                         Semantic Search Engine                        Cultural Heritage Domain    ...
CONTENTUS?    A Teaser3    VISSW 2011 | Johannes Osterhoff | February 13, 2011
4                                          Issue #1                       How to find the needle                         i...
5Image: iStockphoto                     VISSW 2011 | J. Osterhoff | February 13, 2011
6       needle    VISSW 2011 | J. Osterhoff | February 13, 2011
Google Results7    VISSW 2011 | J. Osterhoff | February 13, 2011
Needless to say…8                      needless    VISSW 2011 | J. Osterhoff | February 13, 2011
9                        “needle”    VISSW 2011 | J. Osterhoff | February 13, 2011
10         Jimmy Needles                                                           Image: Jay P. Lee                      ...
11                         “needle”     VISSW 2011 | J. Osterhoff | February 13, 2011
12                 Needles, CA                                          Location                                          ...
13                         “needle”     VISSW 2011 | J. Osterhoff | February 13, 2011
14                         “Needle”                                      Blockbuster                               by John...
15                                           Issue #1               How to find different “needles”                      i...
16                                           Issue #2                              How to present                         ...
Google Results17     VISSW 2011 | J. Osterhoff | February 13, 2011
Flickr Results18     VISSW 2011 | J. Osterhoff | February 13, 2011
YouTube Results19     VISSW 2011 | J. Osterhoff | February 13, 2011
Wikipedia Results20     VISSW 2011 | J. Osterhoff | February 13, 2011
21                                           Issue #2                              How to present                         ...
22                                         Why?     VISSW 2011 | J. Osterhoff | February 13, 2011
23                  30,000+                              Libraries in Germany     VISSW 2011 | J. Osterhoff | February 13,...
24                  30,000+                              Libraries in Germany     VISSW 2011 | J. Osterhoff | February 13,...
Image: Library of Congress25                                      Books     VISSW 2011 | J. Osterhoff | February 13, 2011
26                                    Articles     VISSW 2011 | J. Osterhoff | February 13, 2011
Image: Library of Congress27                                       Video     VISSW 2011 | J. Osterhoff | February 13, 2011
28                                        Audio     VISSW 2011 | J. Osterhoff | February 13, 2011
29                                           Issue #3                    How to make the German                   cultural...
30                                   Authors                            i.e. Bertolt Brecht     VISSW 2011 | J. Osterhoff ...
31                                       Years                                      i.e. 1848     VISSW 2011 | J. Osterhof...
Peoplei.e. from Ulm
Image: Wikipedia (Softeis)33                                 Locations                         among many others…     VISS...
Image: Wikipedia34                              Music Style     VISSW 2011 | J. Osterhoff | February 13, 2011
Process Chain35                                                     HPI Research     VISSW 2011 | J. Osterhoff | February ...
On the Interface36                                        Media Assets                 Books, Articles, Video, Audio      ...
37                                           Issue #3                    How to make the German                    cultura...
38     VISSW 2011 | J. Osterhoff | February 13, 2011
39              1. What is CONTENTUS?              2. Interface Design              3. Demonstration of the System        ...
What is CONTENTUS?     Introduction40      ■ a semantic search engine and a multimedia         retrieval system     VISSW ...
What is CONTENTUS?     Introduction41      ■ a semantic search engine and a multimedia         retrieval system      ■ web...
What is CONTENTUS?     Introduction42      ■ a semantic search engine and a multimedia         retrieval system      ■ web...
What is CONTENTUS?     Introduction43      ■ a semantic search engine and a multimedia         retrieval system      ■ web...
What is CONTENTUS?     Introduction44      ■ Combination of a facetted browsing and         breadcrumb-based navigation   ...
What is CONTENTUS?     Introduction45      “More than       the sum of its parts”     VISSW 2011 | J. Osterhoff | February...
What is CONTENTUS?     Introduction46      More efficient      and more pleasant      user experience.     VISSW 2011 | J....
What is CONTENTUS?     Context47      ■ new technologies for managing,         organizing, and searching multimedia       ...
What is CONTENTUS?     Context48      ■ semantic search technology promises         more accurate and complete search     ...
What is CONTENTUS?     Technologies49      ■ all types of media assets      ■ technologies            media optimization ...
50              1. What is CONTENTUS?              2. Interface Design              3. Demonstration of the System        ...
User Interface     Paradigms51     No research on     a single aspect of     current approaches     in semantic multimedia...
User Interface     Paradigms52     But:     exploiting synergies     by combining     state-of-the-art     technologies in...
User Interface     Paradigms53     semantic     multimedia     technologies     VISSW 2011 | Johannes Osterhoff | February...
User Interface     Paradigms54     semantic +     multimedia     technologies     VISSW 2011 | Johannes Osterhoff | Februa...
User Interface     Paradigms55     semantic +     multimedia     technologies +     interface design     VISSW 2011 | Joha...
User Interface     Paradigms56              combination of multimedia                     and semantic search             ...
User Interface     Paradigms57         compare different media at one glance         how to summarize content most      ...
User Interface     Paradigms58     multimodal     VISSW 2011 | Johannes Osterhoff | February 13, 2011
User Interface     Paradigms59     multimodal     = articles, books,       video, audio, people,       locations on one   ...
User Interface     Paradigms60     The route with the     user‟s experience     gets more important     than the original ...
User Interface     Paradigms61                            main goal            satisfying search experience               ...
62
63   Search Area
64   Search Field   breadcrumbs
65
66
67     Result Area
68                media types                  history                 collection     organize and filter
69      time-line       faceted     suggestions                   refine and filter
search results70
search results71                       article     article            video     audio            location     person      ...
72
73
74
75
76     less important     search results
77
User Interface     Detail Page for Books78               Detail View     VISSW 2011 | Johannes Osterhoff | February 13, 2011
User Interface     Detail Page for Books79                                      full page                                 ...
80              1. What is CONTENTUS?              2. Interface Design              3. Demonstration of the System        ...
81                                      Demo                          http://mediaglobe.yovisto.com:8081/contentus/     VI...
82               1. What is CONTENTUS?               2. Interface Design               3. Demonstration of the System     ...
Summary83                 Issue #1                                    Issue #1         How to find the needle             ...
Future Work84      ■ Feedback that lets the user know why exactly the listed         items have been presented      ■ Poss...
85     VISSW 2011 | Johannes Osterhoff | February 13, 2011
Person                                      Book86             Location                                                   ...
87                              Thank you!                              Johannes P Osterhoff, M.A.                        ...
Upcoming SlideShare
Loading in …5
×

CONTENTUS—A Semantic Multimodal Search User Interface

1,729 views

Published on

Is the main challenge in contemporary information retrieval and semantic search to find the needle
in the haystack? This presentation deals with new issues in this context and demos CONTENTUS, a modern appriach to semantic information retrieval.

There are the slides of the demo at the Visual Interfaces to the Social and Semantic Web Workshop (VISSW 2011) held in conjunction with the International Conference on Intelligent User Interfaces (IUI 2011) on February 13, 2011.

Published in: Technology
  • Be the first to comment

CONTENTUS—A Semantic Multimodal Search User Interface

  1. 1. More than the Sum of its PartsCONTENTUS—A Semantic MultimodalSearch User InterfaceJörg Waitelonis, Johannes Osterhoff, Harald SackVisual Interfaces to the Social and Semantic Web (VISSW 2011)In conjunction with the International Conference on IntelligentUser Interfaces (IUI 2011), J. Osterhoff, February 13, 2011
  2. 2. CONTENTUS? A Teaser2 Semantic Search Engine Cultural Heritage Domain Media types Articles, Books, Video, Audio, … People, Locations, … Part of the HPI research project Semantic Media Explorer VISSW 2011 | Johannes Osterhoff | February 13, 2011
  3. 3. CONTENTUS? A Teaser3 VISSW 2011 | Johannes Osterhoff | February 13, 2011
  4. 4. 4 Issue #1 How to find the needle in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
  5. 5. 5Image: iStockphoto VISSW 2011 | J. Osterhoff | February 13, 2011
  6. 6. 6 needle VISSW 2011 | J. Osterhoff | February 13, 2011
  7. 7. Google Results7 VISSW 2011 | J. Osterhoff | February 13, 2011
  8. 8. Needless to say…8 needless VISSW 2011 | J. Osterhoff | February 13, 2011
  9. 9. 9 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  10. 10. 10 Jimmy Needles Image: Jay P. Lee American basketball coach 1900—1969 Person VISSW 2011 | J. Osterhoff | February 13, 2011
  11. 11. 11 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  12. 12. 12 Needles, CA Location Image: Eddie Biggs VISSW 2011 | J. Osterhoff | February 13, 2011
  13. 13. 13 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  14. 14. 14 “Needle” Blockbuster by John V. Soto from 2010 Film Image: IMDB VISSW 2011 | J. Osterhoff | February 13, 2011
  15. 15. 15 Issue #1 How to find different “needles” in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
  16. 16. 16 Issue #2 How to present “needles” as search results? VISSW 2011 | J. Osterhoff | February 13, 2011
  17. 17. Google Results17 VISSW 2011 | J. Osterhoff | February 13, 2011
  18. 18. Flickr Results18 VISSW 2011 | J. Osterhoff | February 13, 2011
  19. 19. YouTube Results19 VISSW 2011 | J. Osterhoff | February 13, 2011
  20. 20. Wikipedia Results20 VISSW 2011 | J. Osterhoff | February 13, 2011
  21. 21. 21 Issue #2 How to present different “needles” as distinguishable search results on one page? VISSW 2011 | J. Osterhoff | February 13, 2011
  22. 22. 22 Why? VISSW 2011 | J. Osterhoff | February 13, 2011
  23. 23. 23 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
  24. 24. 24 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
  25. 25. Image: Library of Congress25 Books VISSW 2011 | J. Osterhoff | February 13, 2011
  26. 26. 26 Articles VISSW 2011 | J. Osterhoff | February 13, 2011
  27. 27. Image: Library of Congress27 Video VISSW 2011 | J. Osterhoff | February 13, 2011
  28. 28. 28 Audio VISSW 2011 | J. Osterhoff | February 13, 2011
  29. 29. 29 Issue #3 How to make the German cultural heritage available? VISSW 2011 | J. Osterhoff | February 13, 2011
  30. 30. 30 Authors i.e. Bertolt Brecht VISSW 2011 | J. Osterhoff | February 13, 2011
  31. 31. 31 Years i.e. 1848 VISSW 2011 | J. Osterhoff | February 13, 2011
  32. 32. Peoplei.e. from Ulm
  33. 33. Image: Wikipedia (Softeis)33 Locations among many others… VISSW 2011 | J. Osterhoff | February 13, 2011
  34. 34. Image: Wikipedia34 Music Style VISSW 2011 | J. Osterhoff | February 13, 2011
  35. 35. Process Chain35 HPI Research VISSW 2011 | J. Osterhoff | February 13, 2011
  36. 36. On the Interface36 Media Assets Books, Articles, Video, Audio Cultural Categorizations Authors, Year of Creation, People, Locations, Music Styles, Organizations etc. VISSW 2011 | J. Osterhoff | February 13, 2011
  37. 37. 37 Issue #3 How to make the German cultural heritage available in meaningful information retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
  38. 38. 38 VISSW 2011 | J. Osterhoff | February 13, 2011
  39. 39. 39 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  40. 40. What is CONTENTUS? Introduction40 ■ a semantic search engine and a multimedia retrieval system VISSW 2011 | Johannes Osterhoff | February 13, 2011
  41. 41. What is CONTENTUS? Introduction41 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface VISSW 2011 | Johannes Osterhoff | February 13, 2011
  42. 42. What is CONTENTUS? Introduction42 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search VISSW 2011 | Johannes Osterhoff | February 13, 2011
  43. 43. What is CONTENTUS? Introduction43 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search ■ developed in the context of the German research project THESEUS VISSW 2011 | Johannes Osterhoff | February 13, 2011
  44. 44. What is CONTENTUS? Introduction44 ■ Combination of a facetted browsing and breadcrumb-based navigation ■ A time-line enables time-based filtering of search results ■ Personalized search (in the next demonstrator) VISSW 2011 | Johannes Osterhoff | February 13, 2011
  45. 45. What is CONTENTUS? Introduction45 “More than the sum of its parts” VISSW 2011 | J. Osterhoff | February 13, 2011
  46. 46. What is CONTENTUS? Introduction46 More efficient and more pleasant user experience. VISSW 2011 | J. Osterhoff | February 13, 2011
  47. 47. What is CONTENTUS? Context47 ■ new technologies for managing, organizing, and searching multimedia content gets more efficient ■ tremendous diversity of multimedia data affects the retrieval and search process ■ image and video search technology is evolving rapidly VISSW 2011 | Johannes Osterhoff | February 13, 2011
  48. 48. What is CONTENTUS? Context48 ■ semantic search technology promises more accurate and complete search results ■ new user interfaces are required to enable query formulation and intelligent visualization of search results VISSW 2011 | J. Osterhoff | February 13, 2011
  49. 49. What is CONTENTUS? Technologies49 ■ all types of media assets ■ technologies  media optimization  content analysis  semantic metadata generation  speech transcripts  OCR  extraction of semantic entities (i.e. people, places, events) ■ highly heterogeneous collection of semantically annotated data VISSW 2011 | Johannes Osterhoff | February 13, 2011
  50. 50. 50 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  51. 51. User Interface Paradigms51 No research on a single aspect of current approaches in semantic multimedia retrieval… VISSW 2011 | Johannes Osterhoff | February 13, 2011
  52. 52. User Interface Paradigms52 But: exploiting synergies by combining state-of-the-art technologies instead. VISSW 2011 | Johannes Osterhoff | February 13, 2011
  53. 53. User Interface Paradigms53 semantic multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
  54. 54. User Interface Paradigms54 semantic + multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
  55. 55. User Interface Paradigms55 semantic + multimedia technologies + interface design VISSW 2011 | Johannes Osterhoff | February 13, 2011
  56. 56. User Interface Paradigms56 combination of multimedia and semantic search = new challenges in visualizing search results and facilitating navigation VISSW 2011 | J. Osterhoff | February 13, 2011
  57. 57. User Interface Paradigms57  compare different media at one glance  how to summarize content most meaningfully in one single search result item  time-based information (audio, video) VISSW 2011 | J. Osterhoff | February 13, 2011
  58. 58. User Interface Paradigms58 multimodal VISSW 2011 | Johannes Osterhoff | February 13, 2011
  59. 59. User Interface Paradigms59 multimodal = articles, books, video, audio, people, locations on one result-page VISSW 2011 | Johannes Osterhoff | February 13, 2011
  60. 60. User Interface Paradigms60 The route with the user‟s experience gets more important than the original goal of fact finding. VISSW 2011 | Johannes Osterhoff | February 13, 2011
  61. 61. User Interface Paradigms61 main goal satisfying search experience for the user VISSW 2011 | J. Osterhoff | February 13, 2011
  62. 62. 62
  63. 63. 63 Search Area
  64. 64. 64 Search Field breadcrumbs
  65. 65. 65
  66. 66. 66
  67. 67. 67 Result Area
  68. 68. 68 media types history collection organize and filter
  69. 69. 69 time-line faceted suggestions refine and filter
  70. 70. search results70
  71. 71. search results71 article article video audio location person video
  72. 72. 72
  73. 73. 73
  74. 74. 74
  75. 75. 75
  76. 76. 76 less important search results
  77. 77. 77
  78. 78. User Interface Detail Page for Books78 Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
  79. 79. User Interface Detail Page for Books79 full page other pages articles on this page text snippet (OCR) detected entities Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
  80. 80. 80 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  81. 81. 81 Demo http://mediaglobe.yovisto.com:8081/contentus/ VISSW 2011 | Johannes Osterhoff | February 13, 2011
  82. 82. 82 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  83. 83. Summary83 Issue #1 Issue #1 How to find the needle How to find different in the haystack? “needles” in the haystack? Issue #2 Issue #2 How to present How to present “needles” different “needles” as as search results? distinguishable search results? Issue #3 Issue #3 How to make the German How to make the German cultural heritage available? Cultural Heritage available in meaningful Information Retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
  84. 84. Future Work84 ■ Feedback that lets the user know why exactly the listed items have been presented ■ Possibility to connect with friends and to share results with them ■ Suggestions based on friends„ likes ■ State-of-the-art multi-platform web design ■ Next demonstrator to be presented at IBC 2011 Amsterdam VISSW 2011 | Johannes Osterhoff | February 13, 2011
  85. 85. 85 VISSW 2011 | Johannes Osterhoff | February 13, 2011
  86. 86. Person Book86 Location Audio Video VISSW 2011 | Johannes Osterhoff | February 13, 2011
  87. 87. 87 Thank you! Johannes P Osterhoff, M.A. johannes.osterhoff@hpi.uni-potsdam.de Hasso-Plattner-Institute, Prof.-Dr.-Helmert-Str. 2-3, 14482 Potsdam, Germany VISSW 2011 | Johannes Osterhoff | February 13, 2011

×