• Save
A Semantic Multimedia Web (Part 3)
Upcoming SlideShare
Loading in...5
×

Like this? Share it with your network

Share

A Semantic Multimedia Web (Part 3)

  • 4,136 views
Uploaded on

A Semantic Multimedia Web: Part 3 of a series of lectures given at the Universidad Politécnica de Madrid (UPM), May 2009

A Semantic Multimedia Web: Part 3 of a series of lectures given at the Universidad Politécnica de Madrid (UPM), May 2009

More in: Education
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
  • thanq its really good and useful to me in thinking about innovation comes along with think of that.... really good work.... tanq for this.....
    Sharika
    http://financeadded.com http://traveltreble.com
    Are you sure you want to
    Your message goes here
No Downloads

Views

Total Views
4,136
On Slideshare
4,128
From Embeds
8
Number of Embeds
3

Actions

Shares
Downloads
12
Comments
1
Likes
4

Embeds 8

http://www.slideshare.net 6
http://sassou1982.blogspot.com 1
http://health.medicbd.com 1

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. A Semantic Multimedia Web: Create, Annotate, Present and Share your Media Raphaël Troncy, < [email_address] > CWI, Interactive Information Access
  • 2. Agenda
    • The Three Ingredients for Designing Interfaces
    • Semantic Annotation of Multimedia Content
      • Multimedia metadata formats: use cases and requirements
      • Towards an ontology for media objects
    • Interacting with Linked Data
      • Interactive exploration of heterogeneous cultural heritage collections
      • Searching and Browsing Multimedia Semantic Datasets with Cliopatria
      • Towards Configuring Semantic Web Interfaces by Data Mapping
  • 3. The problem
    • Too many users need their own information
      • for their level of expertise
      • using appropriate media
      • in an appropriate style
      • displayed on their own device
    • Multimedia information design is expensive
    • There has to be some automation in the process
  • 4.  
  • 5. Three ingredients Content
  • 6. Content of example Clair-obscur (Frans) en chiaroscuro (Italiaans) betekenen 'licht-donker'. Beide termen worden gebruikt om sterke licht-donkercontrasten in schilderijen, tekeningen en prenten aan te duiden. Hoewel het effect al eerder werd toegepast, is de term pas sinds het einde van de 16de eeuw in zwang. De oorsprong van het woord ligt in Italië. De schilder Caravaggio (1573-1610) maakte het chiaroscuro-effect tot zijn handelsmerk. Hij was een meester in het schilderen van donkere taferelen met één felle lichtbundel .
  • 7. Three ingredients Content Presentation structure
  • 8. Presentation structure of example Clair-obscur (Frans) en chiaroscuro (Italiaans) betekenen 'licht-donker'. Beide termen worden gebruikt om sterke licht-donkercontrasten in schilderijen, tekeningen en prenten aan te duiden. Hoewel het effect al eerder werd toegepast, is de term pas sinds het einde van de 16de eeuw in zwang. De oorsprong van het woord ligt in Italië. De schilder Caravaggio (1573-1610) maakte het chiaroscuro-effect tot zijn handelsmerk. Hij was een meester in het schilderen van donkere taferelen met één felle lichtbundel . Chiaroscuro & Rembrandt Harmensz. Van Rijn description examples title
  • 9. Three ingredients Content Presentation structure Aesthetics abcdefghij abcdefghijklmn ABCDEFGHIJK abcdefghijklm abcdefghijklm
  • 10. Aesthetics of example Colours Fonts Layout Colours
  • 11. Design dependencies Content Presentation structure Aesthetics abcdefghij abcdefghijk ABCDEFGHIJK abcdefghijk abcdefghijk
  • 12. Presentation Structure depends on Layout description title title description description examples title description examples title
  • 13. Style Depends on Content Chiaroscuro & Rembrandt The Stone Bridge (1638) Chiaroscuro & Rembrandt The Stone Bridge (1638)
  • 14. Presentation structure depends on content Rembrandt Caravaggists chiaroscuro 1631 1628 1638 1623 1628
  • 15. Example Presentation Structure Rembrandt Chiaroscuro Caravaggists 1631 1628 1638 1623 1628
  • 16. Agenda
    • The Three Ingredients for Designing Interfaces
    • Semantic Annotation of Multimedia Content
      • Multimedia metadata formats: use cases and requirements
      • Towards an ontology for media objects
    • Interacting with Linked Data
      • Interactive exploration of heterogeneous cultural heritage collections
      • Searching and Browsing Multimedia Semantic Datasets with Cliopatria
      • Towards Configuring Semantic Web Interfaces by Data Mapping
  • 17. Multimedia Web Tagging Semantic Web
  • 18. Making Music Out Of The Social Noise
  • 19. W3C Multimedia Semantics XG (2007) W3C Multimedia Semantics XG http://www.w3.org/2005/Incubator/mmsem/
  • 20. Managing Personal Photos
    • Interoperable Image Metadata
      • Combining EXIF, MPEG-7, IPTC and DIG35 metadata using RDF and OWL schemas
  • 21. Facetting Music Songs
    • Interoperable Music and Social Metadata
      • ID3 Tags + low-level features extraction + lastFM recommendations + FOAF profiles + ...
      • Auto-construction of playlist (similar bit rate), Personalization, Browsing music store
  • 22. Metadata Working Group
  • 23.  
  • 24. W3C Media Annotations WG (2009) http://www.w3.org/2008/WebVideo/Annotations/
  • 25. W3C Media Annotations WG
    • Many standards: EXIF, ID3, MPEG 7, SMIL, iTunes XML, Yahoo! MediaRSS, CableLabs VOD Metadata Content, Video sitemaps, IPTC, EBU, etc.
    • Develop a simple lingua-franca ontology
  • 26. W3C Media Annotations WG Output
    • Use Cases and Requirements for Ontology and API for Media Object 1.0
      • Working Draft, 19 January 2009
    • Giant Table of mapping between multimedia formats with XMP as the pivot format
      • Common Table, 15 April 2009
    • Ontology for Media Objects 1.0
      • Table, 28 April, 2009
  • 27. Agenda
    • The Three Ingredients for Designing Interfaces
    • Semantic Annotation of Multimedia Content
      • Multimedia metadata formats: use cases and requirements
      • Towards an ontology for media objects
    • Interacting with Linked Data
      • Interactive exploration of heterogeneous cultural heritage collections
      • Searching and Browsing Multimedia Semantic Datasets with Cliopatria
      • Towards Configuring Semantic Web Interfaces by Data Mapping
  • 28.  
  • 29. Design interactive interfaces to explore a large heterogeneous graph Presentation @ Talis December 2008
  • 30. Browsing Linked Data Tabulator (CSAIL, MIT)
  • 31. Browsing Linked Data Disco (Free University of Berlin)
  • 32. Browsing Linked Data Marbles (Free University of Berlin)
  • 33. Browsing Linked Data Zitgist (Zitgist LLC)
  • 34. Browsing Linked Data OpenLink Data Explorer (OpenLink Software)
  • 35. Browsing Linked Data (more)
    • RKBExplorer: http://www.rkbexplorer.com/
    • Humboldt: http://humboldt-project.org/
    • Fenfire: http://fenfire.org/
    • Cliopatria: http://e-culture.multimedian.nl/software/ClioPatria.shtml
  • 36. cartoons videos
  • 37. animations blogs
  • 38. News Production
  • 39. News Production: Complex & Dynamic Workflows
  • 40. News Production: As is
    • Example: VRT, a Flemish broadcaster
      • Classical broadcast: linear, 5 broadcasts a day
      • On-demand (“ooit gemist”): replay of the linear broadcasts
      • Web (“deredactie.be”): news clips, continuous updates
    • Linear broadcasts have a rigid structure:
      • Max 20 news items & Max 40 minutes
    • More content available than broadcasted
      • News feeds (EBU/Eurovision, Reuters, …)
      • Produced items never or only a few times broadcasted
    • User’s wishes: enable a news consumer to compose his own news program without interfering with current news production processes
  • 41. Metadata
  • 42. Metadata
    • Information on the news story
      • Content description (who, what, where, when, why, …)
      • Administrative information (reporter, cameraman, …)
      • Rights information (usage terms, copyright statement, …)
      • Technical information (video format, audio format, …)
    • Past: plain text/proprietary structured text (e.g. telex, XML)
    • Present: News-ML / News-ML G2 (XML-based)
    • Future: Semantic News-ML (RDF-based)
  • 43. News Workflow Interoperability
    • No integration of media (stories, photo, animation, video)
    • Little (or no) context in the news presentation
    • Lack of interoperability in the current workflow
    NAR Schema Controlled Vocabularies Broadcaster Schema NewsCodes User Vocabulary
  • 44. Metadata is Key
    • (Ultimate) Goal:
      • Provide an environment for searching and browsing contextualized multimedia news information
    • Required integration:
      • Data: various media, different forms, various sources
      • Metadata: schema integration, semantic models
    • Influence and implications of UI:
      • How to represent semantic multimedia metadata to facilitate presenting information?
      • in other words ... What constraints do end-user interfaces put on the modeling of the metadata?
  • 45. News and Multimedia Formats (NAR) News Architecture NewsML G2 EventsML G2 SportsML G2
  • 46. Porting Schemas and Thesauri to the Semantic Web
    • Methodologies and tools for building ontologies: ... from scratch
    • ʺ SKOSification ʺ of thesauri in the CH domain:
      • preparation, syntactic and semantic conversion, standardization
     Lack of best practices for modeling ontologies from UML diagrams, integrating ontologies with various thesauri, while taking the end-user interface into account
  • 47. Building a Semantic Web Infrastructure for News 1 2 3 4 Modeling the NAR ontology Linking with media ontologies Building SKOS thesauri Enriching the metadata
  • 48. Step 1: Modeling the NAR Ontology  focus on reuse of XML types leading to multiple repetition resulting in overly complex nested XML structures
  • 49. Step 1: Modeling the NAR Ontology
    • Flattening the XML structure
    PhotoNewsItem NewsItem
  • 50. Step 1: Modeling the NAR Ontology
    • Modeling unique identifiers
      • Use of dereferencable URIs for any resources (news items + vocabularies)
      • Future: Use of URIs for resource fragments http://www.youtube.com/watch?v=1bibCui3lFM#t=1m45s
    • Modeling the provenance of the information
      • Reification
      • Named (and Networked) Graphs
    {<> nar:subject cat:11002000} dc:creator team:md ; dc:modified ‘‘2005-11-11T08:00:00Z’’.
  • 51. Step 2: Linking with Media Ontologies foaf:Person ≈ nar:Person dc:Subject ≈ nar:Subject sioc:Item ≈ nar:Item geo:lat geo:long +
  • 52. Step 3: Getting SKOS Vocabularies
  • 53. Step 3: Getting SKOS Vocabularies
  • 54. Step 4: Enriching the News Metadata
    • Concepts/Entities that are subject of news
      • Thematic categories
      • People
      • Organizations
      • Geopolitical Areas
      • Points of Interest
      • Events
      • Products or artefacts
  • 55. Step 4: Enriching the News Metadata NAR Ontology NewsCodes Thesaurus Named Entity Recognition Domain Ontologies
  • 56. Step 4: Enriching the News Metadata NAR Ontology NewsCodes Thesaurus Domain Ontologies Concept Detectors
  • 57. Web of Data and Linked Data dbpedia:Zidane foaf:depicts nar:location geonames:2950159 nar:subject nc:15054000 events:id wp:2006_FIFA_World_Cup#Final
  • 58. Presenting News Information
    • Dimensions used for searching news items
      • When time 10/07/2006
      • Where location Paris
      • What is depicted J. Chirac, Z. Zidane
      • Why event WC 2006
      • Who photographer Bertrand Guay, AFP
    Metadata
  • 59. Semantic Search of Multimedia News Powered by ClioPatria 1.0 alpha 3 1,067,754 Total 1,932 INA Broadcast Video (June/July 2006) 61,311 AFP Photos (June/July 2006) 804,446 AFP News Feed (June/July 2006) 53,468 DBpedia, Geonames 34,903 Thesauri: newscodes 104,358 Domain Specific Ontologies: football 7,336 General Ontologies: NAR, DC, FOAF Number of RDF Triples Description
  • 60.  
  • 61. Provide new dimensions for browsing Distinguish field images vs stadium and street images with a grass detector for the World Cup dataset
  • 62.  
  • 63.  
  • 64.  
  • 65.  
  • 66. Credits
    • Datasets:
    • People:
    • More info: http:// newsml.cwi.nl
  • 67. ABC Event Model
  • 68. CIDOC CRM Event Model
  • 69. CultureSampo Event Model
  • 70. Framenet Event Model
  • 71. Motools Event Model
  • 72. Towards an Linked Events Model?
  • 73.  
  • 74. Configuring Semantic Web Interfaces by Data Mapping Presentation @ VISSW Visual Interfaces to the Social and the Semantic Web February 2009
  • 75. Literature
    • Michiel Hildebrand, Jacco van Ossenbruggen and Lynda Hardman: /facet: A Browser for Heterogeneous Semantic Web Repositories . In 5th International Semantic Web Conference (ISWC'2006) , pages 272-285, Athens (GA), USA, November 5-9, 2006.
    • Jan Wielemaker, Michiel Hildebrand, Jacco van Ossenbruggen and Guus Schreiber: Infrastructure for thesaurus-based search and annotation: evauating the standards . In 7th International Semantic Web Conference (ISWC'2008) , pages ?-?, Karlsruhe, Germany, October 26-30, 2008.
    • Raphaël Troncy: Bringing the IPTC News Architecture into the Semantic Web . In 7th International Semantic Web Conference (ISWC'2008) , pages 483-498, Karlsruhe, Germany, October 26-30, 2008.
  • 76. Assignment (follow-up)
  • 77. Assignment (task)
    • Build your ontology
      • Pick a domain
      • Choose its relevant concepts and properties
      • Sketch a first model (draw on paper, using online tool)
      • Search for existing concepts and properties and refine your ontology by asserting equivalences: link your model!
        • Use Semantic Web Search Engines
      • Populate your ontology with samples from the Linked Data Cloud
        • Look up entities from the cloud
    • Report your experience
      • Send the ontology and the knowledge base: owl, rdf, skos files
        • Check for their validity: visualize the data in some tools
      • Write a short report (txt, html) summarizing the whole process
  • 78. What are the messages?
    • Features can be extracted and used to describe multimedia content (metadata)
    • Multimedia presentations embody messages
    • Media, structure and aesthetics all contribute to conveying the message
    • The message can be made explicit (more metadata)
    • Media content and metadata can be passed around and among systems
    • We need to agree on how to do this (e.g. canonical processes; COMM; W3C working groups; IPTC)
    • Users can be given much richer and more flexible access to (semantically annotated) content, but … ...we are still figuring out how to do this.
  • 79. Thanks for your attention http:// www.cwi.nl/~troncy