CONTENTUS—A Semantic Multimodal Search User Interface

1,526 views

Published on

Is the main challenge in contemporary information retrieval and semantic search to find the needle
in the haystack? This presentation deals with new issues in this context and demos CONTENTUS, a modern appriach to semantic information retrieval.

There are the slides of the demo at the Visual Interfaces to the Social and Semantic Web Workshop (VISSW 2011) held in conjunction with the International Conference on Intelligent User Interfaces (IUI 2011) on February 13, 2011.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,526
On SlideShare
0
From Embeds
0
Number of Embeds
38
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

CONTENTUS—A Semantic Multimodal Search User Interface

  1. 1. More than the Sum of its PartsCONTENTUS—A Semantic MultimodalSearch User InterfaceJörg Waitelonis, Johannes Osterhoff, Harald SackVisual Interfaces to the Social and Semantic Web (VISSW 2011)In conjunction with the International Conference on IntelligentUser Interfaces (IUI 2011), J. Osterhoff, February 13, 2011
  2. 2. CONTENTUS? A Teaser2 Semantic Search Engine Cultural Heritage Domain Media types Articles, Books, Video, Audio, … People, Locations, … Part of the HPI research project Semantic Media Explorer VISSW 2011 | Johannes Osterhoff | February 13, 2011
  3. 3. CONTENTUS? A Teaser3 VISSW 2011 | Johannes Osterhoff | February 13, 2011
  4. 4. 4 Issue #1 How to find the needle in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
  5. 5. 5Image: iStockphoto VISSW 2011 | J. Osterhoff | February 13, 2011
  6. 6. 6 needle VISSW 2011 | J. Osterhoff | February 13, 2011
  7. 7. Google Results7 VISSW 2011 | J. Osterhoff | February 13, 2011
  8. 8. Needless to say…8 needless VISSW 2011 | J. Osterhoff | February 13, 2011
  9. 9. 9 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  10. 10. 10 Jimmy Needles Image: Jay P. Lee American basketball coach 1900—1969 Person VISSW 2011 | J. Osterhoff | February 13, 2011
  11. 11. 11 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  12. 12. 12 Needles, CA Location Image: Eddie Biggs VISSW 2011 | J. Osterhoff | February 13, 2011
  13. 13. 13 “needle” VISSW 2011 | J. Osterhoff | February 13, 2011
  14. 14. 14 “Needle” Blockbuster by John V. Soto from 2010 Film Image: IMDB VISSW 2011 | J. Osterhoff | February 13, 2011
  15. 15. 15 Issue #1 How to find different “needles” in the haystack? VISSW 2011 | J. Osterhoff | February 13, 2011
  16. 16. 16 Issue #2 How to present “needles” as search results? VISSW 2011 | J. Osterhoff | February 13, 2011
  17. 17. Google Results17 VISSW 2011 | J. Osterhoff | February 13, 2011
  18. 18. Flickr Results18 VISSW 2011 | J. Osterhoff | February 13, 2011
  19. 19. YouTube Results19 VISSW 2011 | J. Osterhoff | February 13, 2011
  20. 20. Wikipedia Results20 VISSW 2011 | J. Osterhoff | February 13, 2011
  21. 21. 21 Issue #2 How to present different “needles” as distinguishable search results on one page? VISSW 2011 | J. Osterhoff | February 13, 2011
  22. 22. 22 Why? VISSW 2011 | J. Osterhoff | February 13, 2011
  23. 23. 23 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
  24. 24. 24 30,000+ Libraries in Germany VISSW 2011 | J. Osterhoff | February 13, 2011
  25. 25. Image: Library of Congress25 Books VISSW 2011 | J. Osterhoff | February 13, 2011
  26. 26. 26 Articles VISSW 2011 | J. Osterhoff | February 13, 2011
  27. 27. Image: Library of Congress27 Video VISSW 2011 | J. Osterhoff | February 13, 2011
  28. 28. 28 Audio VISSW 2011 | J. Osterhoff | February 13, 2011
  29. 29. 29 Issue #3 How to make the German cultural heritage available? VISSW 2011 | J. Osterhoff | February 13, 2011
  30. 30. 30 Authors i.e. Bertolt Brecht VISSW 2011 | J. Osterhoff | February 13, 2011
  31. 31. 31 Years i.e. 1848 VISSW 2011 | J. Osterhoff | February 13, 2011
  32. 32. Peoplei.e. from Ulm
  33. 33. Image: Wikipedia (Softeis)33 Locations among many others… VISSW 2011 | J. Osterhoff | February 13, 2011
  34. 34. Image: Wikipedia34 Music Style VISSW 2011 | J. Osterhoff | February 13, 2011
  35. 35. Process Chain35 HPI Research VISSW 2011 | J. Osterhoff | February 13, 2011
  36. 36. On the Interface36 Media Assets Books, Articles, Video, Audio Cultural Categorizations Authors, Year of Creation, People, Locations, Music Styles, Organizations etc. VISSW 2011 | J. Osterhoff | February 13, 2011
  37. 37. 37 Issue #3 How to make the German cultural heritage available in meaningful information retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
  38. 38. 38 VISSW 2011 | J. Osterhoff | February 13, 2011
  39. 39. 39 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  40. 40. What is CONTENTUS? Introduction40 ■ a semantic search engine and a multimedia retrieval system VISSW 2011 | Johannes Osterhoff | February 13, 2011
  41. 41. What is CONTENTUS? Introduction41 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface VISSW 2011 | Johannes Osterhoff | February 13, 2011
  42. 42. What is CONTENTUS? Introduction42 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search VISSW 2011 | Johannes Osterhoff | February 13, 2011
  43. 43. What is CONTENTUS? Introduction43 ■ a semantic search engine and a multimedia retrieval system ■ web-based graphical user interface ■ content-based suggestions and a multi-modal presentation of search results to support semantic search ■ developed in the context of the German research project THESEUS VISSW 2011 | Johannes Osterhoff | February 13, 2011
  44. 44. What is CONTENTUS? Introduction44 ■ Combination of a facetted browsing and breadcrumb-based navigation ■ A time-line enables time-based filtering of search results ■ Personalized search (in the next demonstrator) VISSW 2011 | Johannes Osterhoff | February 13, 2011
  45. 45. What is CONTENTUS? Introduction45 “More than the sum of its parts” VISSW 2011 | J. Osterhoff | February 13, 2011
  46. 46. What is CONTENTUS? Introduction46 More efficient and more pleasant user experience. VISSW 2011 | J. Osterhoff | February 13, 2011
  47. 47. What is CONTENTUS? Context47 ■ new technologies for managing, organizing, and searching multimedia content gets more efficient ■ tremendous diversity of multimedia data affects the retrieval and search process ■ image and video search technology is evolving rapidly VISSW 2011 | Johannes Osterhoff | February 13, 2011
  48. 48. What is CONTENTUS? Context48 ■ semantic search technology promises more accurate and complete search results ■ new user interfaces are required to enable query formulation and intelligent visualization of search results VISSW 2011 | J. Osterhoff | February 13, 2011
  49. 49. What is CONTENTUS? Technologies49 ■ all types of media assets ■ technologies  media optimization  content analysis  semantic metadata generation  speech transcripts  OCR  extraction of semantic entities (i.e. people, places, events) ■ highly heterogeneous collection of semantically annotated data VISSW 2011 | Johannes Osterhoff | February 13, 2011
  50. 50. 50 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  51. 51. User Interface Paradigms51 No research on a single aspect of current approaches in semantic multimedia retrieval… VISSW 2011 | Johannes Osterhoff | February 13, 2011
  52. 52. User Interface Paradigms52 But: exploiting synergies by combining state-of-the-art technologies instead. VISSW 2011 | Johannes Osterhoff | February 13, 2011
  53. 53. User Interface Paradigms53 semantic multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
  54. 54. User Interface Paradigms54 semantic + multimedia technologies VISSW 2011 | Johannes Osterhoff | February 13, 2011
  55. 55. User Interface Paradigms55 semantic + multimedia technologies + interface design VISSW 2011 | Johannes Osterhoff | February 13, 2011
  56. 56. User Interface Paradigms56 combination of multimedia and semantic search = new challenges in visualizing search results and facilitating navigation VISSW 2011 | J. Osterhoff | February 13, 2011
  57. 57. User Interface Paradigms57  compare different media at one glance  how to summarize content most meaningfully in one single search result item  time-based information (audio, video) VISSW 2011 | J. Osterhoff | February 13, 2011
  58. 58. User Interface Paradigms58 multimodal VISSW 2011 | Johannes Osterhoff | February 13, 2011
  59. 59. User Interface Paradigms59 multimodal = articles, books, video, audio, people, locations on one result-page VISSW 2011 | Johannes Osterhoff | February 13, 2011
  60. 60. User Interface Paradigms60 The route with the user‟s experience gets more important than the original goal of fact finding. VISSW 2011 | Johannes Osterhoff | February 13, 2011
  61. 61. User Interface Paradigms61 main goal satisfying search experience for the user VISSW 2011 | J. Osterhoff | February 13, 2011
  62. 62. 62
  63. 63. 63 Search Area
  64. 64. 64 Search Field breadcrumbs
  65. 65. 65
  66. 66. 66
  67. 67. 67 Result Area
  68. 68. 68 media types history collection organize and filter
  69. 69. 69 time-line faceted suggestions refine and filter
  70. 70. search results70
  71. 71. search results71 article article video audio location person video
  72. 72. 72
  73. 73. 73
  74. 74. 74
  75. 75. 75
  76. 76. 76 less important search results
  77. 77. 77
  78. 78. User Interface Detail Page for Books78 Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
  79. 79. User Interface Detail Page for Books79 full page other pages articles on this page text snippet (OCR) detected entities Detail View VISSW 2011 | Johannes Osterhoff | February 13, 2011
  80. 80. 80 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  81. 81. 81 Demo http://mediaglobe.yovisto.com:8081/contentus/ VISSW 2011 | Johannes Osterhoff | February 13, 2011
  82. 82. 82 1. What is CONTENTUS? 2. Interface Design 3. Demonstration of the System 4. Summary VISSW 2011 | J. Osterhoff | February 13, 2011
  83. 83. Summary83 Issue #1 Issue #1 How to find the needle How to find different in the haystack? “needles” in the haystack? Issue #2 Issue #2 How to present How to present “needles” different “needles” as as search results? distinguishable search results? Issue #3 Issue #3 How to make the German How to make the German cultural heritage available? Cultural Heritage available in meaningful Information Retrieval? VISSW 2011 | J. Osterhoff | February 13, 2011
  84. 84. Future Work84 ■ Feedback that lets the user know why exactly the listed items have been presented ■ Possibility to connect with friends and to share results with them ■ Suggestions based on friends„ likes ■ State-of-the-art multi-platform web design ■ Next demonstrator to be presented at IBC 2011 Amsterdam VISSW 2011 | Johannes Osterhoff | February 13, 2011
  85. 85. 85 VISSW 2011 | Johannes Osterhoff | February 13, 2011
  86. 86. Person Book86 Location Audio Video VISSW 2011 | Johannes Osterhoff | February 13, 2011
  87. 87. 87 Thank you! Johannes P Osterhoff, M.A. johannes.osterhoff@hpi.uni-potsdam.de Hasso-Plattner-Institute, Prof.-Dr.-Helmert-Str. 2-3, 14482 Potsdam, Germany VISSW 2011 | Johannes Osterhoff | February 13, 2011

×