Mediaglobe & Contentus - from 10.000 Feet Above Ground

1,503 views

Published on

Presentation from my Research talk, Oct 5, 2010 on our 2 research projects MEDIAGLOBE and CONTENTUS within the German THESEUS research programme.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,503
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
17
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Mediaglobe & Contentus - from 10.000 Feet Above Ground

  1. 1. Mediaglobe & CONTENTUS from 10.000 feet above ground Harald Sack Internet Technologies and Systems (ITS) Future Internet Technologies / Semantic Technologies Hasso-Plattner-Institute for IT Systems Engineering Research Seminar Oct 5th, 2010
  2. 2. Mediaglobe & Contentus from 10.000 feet above ground 2 • Semantic Technologies & Multimedia Retrieval • Theseus Research Program • Projekt Mediaglobe • Projekt Theseus/Contentus Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  3. 3. Semantic Technologies & Multimedia Retrieval 3 • 2009/01 started with 1 senior researcher ... • 2009/03 Jörg Waitelonis • 2009/12 Zalan Kramer • 2010/01 Johannes Hercher • 2010/03 Bernhard Quehl • 2010/03 Haojin Yang • 2010/05 Nadine Ludwig, Johannes Osterhoff • 2010/07 Magnus Knuth • 2010/09 Joscha Jäger • 2010/11 N.N. Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  4. 4. Semantic Technologies & Multimedia Retrieval 4 • Research Topics • Semantic Web Technologies • Ontological Engineering • Information Retrieval • Multimedia Retrieval • Multimedia Analysis • Social Networking • Data/Information Visualization Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  5. 5. Semantic Technologies & Multimedia Retrieval 5 • Research Projects Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  6. 6. Mediaglobe & Contentus from 10.000 feet above ground 6 • Semantic Technologies & Multimedia Retrieval • Theseus Research Program • Project Mediaglobe • Project Theseus/Contentus Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  7. 7. Theseus Research Program 7 • THESEUS - New Technologies for the Internet of Services • GOAL: to develop a new Internet-based infrastructure in order to better use and utilize the knowledge available on the Internet. • FOCUS: Computational Linguistics and Semantic Technologies • Overall Budget: 200 Mio Euro / Time Frame: 2007 - 2012 • Partners: antibodies-onlinedefa-spektrum GmbH / Deutsche Thomson oHG / DISY InformationssystemeCIM Aachen GmbH / GmbH / Averbis GmbH / B2M Software AG / Blue Order Technologies AG / GmbH / Empolis GmbH / EXAPT Systemtechnik GmbH / Festo AG & Co. KG / Festool GmbH / Fraunhofer-Gesellschaft / German National Library / German Research Center for Artifi cial Intelligence (DFKI) / Hasso-Plattner-Institut für Softwaresystemtechnik (HPI) GmbH / Hessian Telemedia Technology Competence Center (httc e.V.) / imc information multimedia communication AG / InfoChem Gesellschaft für chemische Information mbH / Infoman AG / Institut für Rundfunktechnik GmbH / intelligent views gmbh / jCOM1 AG / Karlsruhe Institute of Technology (KIT) / Ligmatech Automationssysteme GmbH / Ludwig-Maximilians-Universität (LMU) / Medien Bildungsgesellschaft Babelsberg GmbH / Metris GmbH / mufi n GmbH / neofonie GmbH / ontoprise GmbH / raumobil GmbH / Research Center for Information Technology Karlsruhe (FZI) / RESprotect GmbH / RWTH Aachen University / SAP AG / SEEBURGER AG / Siemens AG / Sterling SIHI GmbH / Technische Universität Darmstadt / Technische Universität Dresden / Technische Universität München / Transinsight GmbH / Universität des Saarlandes / Universität Freiburg / Universität Karlsruhe (TH) / Universität Leipzig / Universität Stuttgart / Universitätsklinikum Erlangen / VDMA – Verband Deutscher Maschinen- und Anlagenbau e.V. / Yellowmap AG www.theseus-programm.de Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  8. 8. Theseus Research Program 8 Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  9. 9. Theseus Research Program 9 THESEUS Core Technology Cluster • WP1: CTC Management (HHI) • WP2: Video, Audio, Metadata, Platforms (HHI) • WP3: Ontology Management (FZI) • WP4: Semantic Access to Media and Services (DFKI) • WP5: User Interface, Visualization (IGD) • WP6: Statistical Machine Learning (Siemens) • WP7: DRM/IPR Management (IIS) • WP8: Evaluation (IDMT) THESEUS Use Cases • ALEXANDRIA - A Knowledge Platform on the Internet • CONTENTUS - Technologies for the Library of the Future • MEDICO - Intelligent Searches in Medical Databases • ORDO - Order in a Digital World • PROCESSUS - Making Better Use of Corporate Knowledge • TEXO - An Infrastructure for Web-Based Services THESEUS SME 2009 • MEDIAGLOBE + 11 other projects Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  10. 10. Mediaglobe & Contentus from 10.000 feet above ground 10 • Semantic Technologies & Multimedia Retrieval • Theseus Research Program • Project Mediaglobe • Project Theseus/Contentus Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  11. 11. Project Mediaglobe - About 11 • THESEUS SME Project • Affiliated with THESEUS/CONTENTUS • Sept 2009 – Aug 2011 / to be extended until June 2012 • 4 Partners / Budget: 2.5 Mio € • Topic • Open Up Audiovisual Media Archives with historic & documentary content • Enable exploratory and semantic search in Audiovisual Media Archives • Business Cases • Semantic Search Engine Infrastructure and Services for • Media Archives, • Broadcasters and Producers www.projekt-mediaglobe.de Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  12. 12. Project Mediaglobe - Partners 12 Project Management Research & Development AV Archive Media Asset Management System Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  13. 13. Project Mediaglobe - Topics 13 Media  Archive  Requirements Digi1za1on  of  AV  Media Rights  Management Automated  Media  Analysis Metadata  Engineering Seman1c  Search User  Interface  Design Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  14. 14. Project Mediaglobe - Topics 14 Topic: Requirement Analysis and Media Census Data Collection from > 200 AV-Archives in Germany about digitization, online distribution, and rights management Topic: Efficient Digitization of AV-Archives Workflow definition and avaluation, best practices Topic: Software Enabled Digital Rights Management Workflow definition and best practices for unique determination of copyrights Topic: automated AV Media Analysis Extraction of textual and semantic metadata for semantic search Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  15. 15. Project Mediaglobe - Topics 15 Topic: Metadata Engineering Definition, interlinking and validation of (semantic) metadata model for media archives Topic: Semantic Search Combining semantic metadata of heterogeneous provenance into semantic search Index to enable high precision/recall multimedia retrieval and exploratory search Topic: User Interface Design Support of innovative search strategies with semantic data/information visualization Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  16. 16. Project Mediaglobe - Responsibilities 16 Media Asset Management Distribution Structural AV-Segmentation Ontology Design Intelligent Character Recognition Entity-Mapping / Schema Mapping Face/Body Detection Semantic Enabled Retrieval Genre Detection Exploratory Search Speaker Detektion GUI Design Automated Speech Recognition Data/Information Visualization Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  17. 17. Project Mediaglobe - HPI Research 17        UIMA  -­‐  Unstructured  Informa1on  Management  Architecture Automated Seman1c Media  Analysis Analysis Structural  Analysis Context  Analysis Media  Transcoding Persistent  Storage Intelligent  Character En1ty  Mapping Recogni1on Face  Detec(on  +  Tracking digi1zed AV-­‐Media Audio  Analysis Genre  Analysis Evalua1on  Framework Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  18. 18. Project Mediaglobe - HPI Research 18 Media Transcoding Archival and Distribution Processing •SD - DVCpro 50 •MPEG4/AVC •HD - DVCpro HD •Downscaling Evaluation Framework •Accurate manual annotation of 25 video clips (750 min) from defa spektrum archive •TREC video test datasets Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  19. 19. Project Mediaglobe - HPI Research 19 Structural Analysis video scenes shots subhots frames Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  20. 20. Project Mediaglobe - HPI Research 20 Structural Analysis • Shot Boundary Detection shots • Identification of • Hard Cuts • Drop Outs histogram differences • Soft Cuts, as e.g., Dissolve, Wipe, Cross-Fade, etc. Analytical Shot Boundary Detection • Analysis of Luminance/Chrominance Histograms • Analysis of Edge Distribution • Analysis of Motion Vectors Machine Learning • Classification of Hard/Soft Cuts based on Image Features • Random Trees • Support Vector Machines Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  21. 21. Project Mediaglobe - HPI Research 21 Structural Analysis Analytical Shot Boundary Detection • How to differentiate between Soft Cuts and Camera Rotation, Pan, and Zoom? • Analysis of Motion Vectors Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  22. 22. Project Mediaglobe - HPI Research 22 Structural Analysis (Preliminary) Evaluation • Yovisto/Mediaglobe • CTC 2 - Shot Detection (HHI) • Advene Shot Detection • Student seminar project (analytical analysis, AL) • Student seminar project (machine learning, ML) recall precision f1 measure yovisto/mediaglobe 0,76 0,77 0,75 Advene 0,64 0,76 0,67 HHI 0,78 0,77 0,77 Students AL 0,72 0,78 0,71 Students ML 0,80 0,81 0,80 new 0,87 0,83 0,85 Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  23. 23. Project Mediaglobe - HPI Research 23 Intelligent Character Recognition • Preprocessing • Keyframe extraction • Script identification • Script filtering • Adaption of script geometry (Deskew) • Image quality enhancement • Optical Character Recognition (OCR) • with standard software (tesseract) • Postprocessing • Keyterm spotting • Lexical analysis Prof. Rudolf Agsten LDPD • Statistical filtering Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  24. 24. Project Mediaglobe - HPI Research 24 Intelligent Character Recognition (a) Original (b) DCT (c) Weighted DCT (d) Normalized (e) Binarized (f) Mask after erosion & dilation Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  25. 25. Project Mediaglobe - HPI Research 25 Intelligent Character Recognition (h) sequence 1 (i) sequence 2 (j) Adapted sequence 1 (k) Adapted sequence 2 Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  26. 26. Project Mediaglobe - HPI Research 26 Metadaten Engineering • Requirement Analysis • Semantic Data Modelling • Vocabulary Inter-Linking • MPEG-7 Compliance Tex Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  27. 27. Project Mediaglobe - HPI Research 27 Metadaten Engineering • Entity Mapping • Mapping keyterms (text) to semantic entities • Context Analysis and Disambiguation ? Truman Capote ? Harry S. Truman Truman User Tag ? Truman, Minesota ? The Truman Show LOD Cloud Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  28. 28. Project Mediaglobe - HPI Research 28 Metadaten Engineering • Entity Mapping • Mapping keyterms (text) to semantic entities • Context Analysis and Disambiguation Truman Eisenhower Potsdam Inauguration Context Graph Analysis Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  29. 29. Project Mediaglobe - HPI Research Automated Media Analysis 29 Semantic Search • Creation of a Semantic Search Index • Query String Mapping and Refinement • Facetted Search • Search by Timeline • Geographical Search • Exploratory Search Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  30. 30. Project Mediaglobe - HPI Research 30 User Interface Design Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  31. 31. Project Mediaglobe - HPI Research 31 User Interface Design Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  32. 32. Project Mediaglobe - HPI Research 32 User Interface Design Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  33. 33. Mediaglobe & Contentus from 10.000 feet above ground 33 • Semantic Technologies & Multimedia Retrieval • Theseus Research Program • Project Mediaglobe • Project Theseus/Contentus Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  34. 34. CONTENTUS 34 • Use Case (among 5 others) of the German Theseus Research Program • Time Frame: 2007 - 2012 • 7 Project Partners • Supported by the Bundesministerium für Wirtschaft und Technologie Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  35. 35. 35 Motivation • Deterioration of Media (Books, Video, Records, DVD, CD… ) • Enormous amount of multimedia objects • High costs and manpower to drive a digitizing workflow • Almost no internet-based linking of cultural goods Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  36. 36. 36 Project Goals • Development of concepts and Technologies for ,Next Generation Multimedia Libraries‘ • Automatic quality control & restauration • Automatic metadata generation • Semi-automatic semantic linking • Incorporation of social networks and expert communities Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  37. 37. 37 Contentus Process Chain HPI Research Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  38. 38. 38 Contentus Service Platform Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  39. 39. 39 Contentus Process Chain Backend Media Processing Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  40. 40. 40 Selected Contentus Components Face Detection / Dirt Detection & Removal Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  41. 41. 41 Selected Contentus Components Face Detection / Scratch Detection & Removal Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  42. 42. 42 Selected Contentus Components Layout Detection / OCR Preprocessing Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  43. 43. 43 Selected Contentus Components Audio Analysis / Audio Annotation Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  44. 44. 44 Contentus SMMS Process Chain Frontend Backend Media Processing Processing Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  45. 45. SMMS GUI DEMO - D2 45 Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam
  46. 46. Mediaglobe & Contentus from 10.000 feet above ground 46 • Semantic Technologies & Multimedia Retrieval • Project Mediaglobe • Project Theseus/Contentus Thank you for your Attention! Dr. Harald Sack, Mediaglobe & Contentus, Research Seminar, 5. Oct. 2010, Hasso-Plattner Institute for IT Systems Engineering, Potsdam

×