• Save
The importance of Linked Media to the Future Web
Upcoming SlideShare
Loading in...5
×
 

The importance of Linked Media to the Future Web

on

  • 3,193 views

If the future Web will be able to fully leverage the scale and quality of online media, a Web scale layer of structured, interlinked media annotations is needed, which we will call Linked Media, ...

If the future Web will be able to fully leverage the scale and quality of online media, a Web scale layer of structured, interlinked media annotations is needed, which we will call Linked Media, inspired by the Linked Data movement for making structured, interlinked descriptions of resources better available online. Mobile and tablet devices, as well as connected TVs, introduce novel application domains that will benefit from broad understanding and acceptance of Linked Media standards. In this talk, I will provide an overview of current practices and specification efforts in the domain of video and Web content integration, drawing from the LinkedTV and MediaMixer projects. From this, I will present a vision for a Linked Media layer on the future Web will can empower new media-centric applications in a world of ubiquitous online multimedia.

Statistics

Views

Total Views
3,193
Views on SlideShare
1,027
Embed Views
2,166

Actions

Likes
2
Downloads
0
Comments
0

3 Embeds 2,166

http://www.linkedtv.eu 1919
http://www.scoop.it 235
https://twitter.com 12

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

The importance of Linked Media to the Future Web The importance of Linked Media to the Future Web Presentation Transcript

  • The importance of Linked Media to theThe importance of Linked Media to theFuture WebFuture WebLyndon NixonLyndon Nixonlyndon.nixon@sti2.orglyndon.nixon@sti2.org@lyndonjbnixon@lyndonjbnixon
  • The Future Web?The Future Web?Our first assumption is that the FutureOur first assumption is that the FutureMedia Internet will involve:Media Internet will involve:•Ubiquituous online, high bandwidthUbiquituous online, high bandwidthaccess via multiple devicesaccess via multiple devices•All our media is digital and stored in andAll our media is digital and stored in andaccessed from the cloudaccessed from the cloud•Media access is seamless and abstractedMedia access is seamless and abstractedfrom its sourcefrom its source•Consumers expect interactivity in andConsumers expect interactivity in andintegration of mediaintegration of mediaImage from www.sics.se/efraim used without permission.The EFRAIM project aims to „identify and develop distributed networksolutions for media distribution“
  • Tom sees a street posterTom sees a street poster•An app with text recognition capability extracts theAn app with text recognition capability extracts theartist name and concert location and via an eventartist name and concert location and via an eventcalendar can determine the event date.calendar can determine the event date.•Tom sees video clips of the artists last concert.Tom sees video clips of the artists last concert.•A map is overlaid showing the location of theA map is overlaid showing the location of theadvertised concert. Photos show Tom that inside itadvertised concert. Photos show Tom that inside ithas a nice industrial architecture.has a nice industrial architecture.•Its a future event, so the ticket can still be boughtIts a future event, so the ticket can still be boughtonline.online.•As he leaves, he can stream audio of the artistsAs he leaves, he can stream audio of the artistssongs.songs.
  • Artist VenueEventVideo (fragments)Event MapPhotosAudioTicketingserviceDate TimeImage (Poster)Spatial regionsearlier than
  • Is it possible today?Is it possible today?•Concept identifiersConcept identifiers•Web APIs using labels (strings) OR internal identifiers, Scarce sharedWeb APIs using labels (strings) OR internal identifiers, Scarce sharedidentifiers across data and media sourcesidentifiers across data and media sources•Heterogeneous APIsHeterogeneous APIs•No machine processable descriptions of online APIs, Programmers needNo machine processable descriptions of online APIs, Programmers needto define API calls, cant be made on the flyto define API calls, cant be made on the fly•Media assets are hidden from computer agentsMedia assets are hidden from computer agents•Media descriptions non-existent / limited / non-standardised in bothMedia descriptions non-existent / limited / non-standardised in bothdata model and vocabulary used, not re-using our concept identifiersdata model and vocabulary used, not re-using our concept identifiers
  • Linked Media PrinciplesLinked Media Principles1.1. Web media descriptions need a common representation ofWeb media descriptions need a common representation ofmedia structuremedia structure2.2. Web media descriptions need a common representation ofWeb media descriptions need a common representation ofmediamedia contentcontent3.3. Web media descriptions need to use a media ontology whichWeb media descriptions need to use a media ontology whichsupportssupports description of both the structure and content of mediadescription of both the structure and content of media4.4. The descriptions of media in terms of common representationsThe descriptions of media in terms of common representationsofof structure and content are the basis for deriving links acrossstructure and content are the basis for deriving links acrossmedia on the Web (Linked Media)media on the Web (Linked Media)
  • State of the ResearchState of the Research
  • Media descriptionMedia description•Metadatas main purpose for media isMetadatas main purpose for media isindexing and retrievalindexing and retrieval•Descriptive metadata for mediaDescriptive metadata for mediacharacteristicscharacteristics•Provenance metadata to credit source andProvenance metadata to credit source andspecify rightsspecify rights•Conceptual metadata to reflect what mediaConceptual metadata to reflect what mediais perceived to representis perceived to represent
  • Media description modelsMedia description models•Highly heterogeneous: varying coverageHighly heterogeneous: varying coveragedepending on originating communitydepending on originating community•W3C Media OntologyW3C Media Ontology defines a shareddefines a sharedsubset of all metadata properties withsubset of all metadata properties with(rough) mappings(rough) mappings•More a basis for metadataMore a basis for metadatainteroperability than a metadata schemainteroperability than a metadata schemato be used directlyto be used directly
  • Conceptual representation of mediaConceptual representation of media•Conceptual annotation is limited in media metadataConceptual annotation is limited in media metadata•How much was the Semantics DS used in MPEG-7?How much was the Semantics DS used in MPEG-7?•Typically whole media items are annotated with labels (strings) or aTypically whole media items are annotated with labels (strings) or alocal controlled vocabulary (examples from EPG data: title, description,local controlled vocabulary (examples from EPG data: title, description,tags, categories, genres)tags, categories, genres)•Media Ontology has only the „keyword“ property to connect media toMedia Ontology has only the „keyword“ property to connect media toconcepts that it describes & does not enforce any means to identify theconcepts that it describes & does not enforce any means to identify theconcepts referred toconcepts referred to
  • Concept spaceConcept space•Globally unambiguous identifiers forGlobally unambiguous identifiers forany concept neededany concept needed•Ideally more information about theIdeally more information about theconcept itself is available via theconcept itself is available via theidentifieridentifier•Use of Web URIs →Use of Web URIs →Linked DataLinked Dataconcept spaceconcept space•Would allow for inference of conceptWould allow for inference of concepttype and relationship to othertype and relationship to otherconceptsconcepts
  • Media fragmentsMedia fragments•The subject of media description isThe subject of media description istypically the whole media assettypically the whole media asset•Parts of media can represent distinctParts of media can represent distinctconcepts, but how to refer to them?concepts, but how to refer to them?•MPEG-7 provided for mediaMPEG-7 provided for mediasegmentation but wasnt well definedsegmentation but wasnt well defined(how does an external application refer(how does an external application referto a media segment?)to a media segment?)•W3C Media Fragment URI SpecificationW3C Media Fragment URI Specificationdefines an agreed syntax for spatial anddefines an agreed syntax for spatial andtemporal parts of audio-visualtemporal parts of audio-visualmaterials on the Webmaterials on the Web
  • Linked Media ontologiesLinked Media ontologiesAnnotationConceptKeywordBBC Ontology +SchemaDotOrgTVANALYSIS RESULTS (Support for segmentation)Ontology forMedia Resources(W3C)Ontology for MediaResources (W3C)Open AnnotationCore Data ModelEXTERNAL DATASETSEntityNERDProvenanceOntology forProvenanceManagementProgrammeBrandSeriesEpisodeVersion BroadcastServiceBroadcast ChannelSceneShotMediaFragmentFacehttp://data.linkedtv.eu/ontologies/core http://www.connectme.at/ontology
  • Media analysisMedia analysis•Video analysis can be used to detectVideo analysis can be used to detectdistinct shots or objects in the videodistinct shots or objects in the video•Combining other feature analysisCombining other feature analysiscan help improve detectioncan help improve detectionresults (results (hybrid approacheshybrid approaches))•Defining what constitutes aDefining what constitutes a„conceptually“ distinct fragment„conceptually“ distinct fragmentis a challenging taskis a challenging task
  • Entity recognitionEntity recognition•Metadata in textual form can beMetadata in textual form can beused in extracting distinct namedused in extracting distinct namedentities, cf.entities, cf. NERDNERD•Subtitles, transcripts, OCR,Subtitles, transcripts, OCR,speech to text...speech to text...•Entity disambiguation is anEntity disambiguation is animportant contributionimportant contributionScreenshot of NERD http://nerd.eurecom.fr
  • Media metadata publicationMedia metadata publication•We need to be able to publishWe need to be able to publishLinked Media following the sameLinked Media following the sameprinciples as Linked Dataprinciples as Linked Data•The media asset has a globallyThe media asset has a globally(Web-wide) unique identifier(Web-wide) unique identifier•Metadata about the media assetMetadata about the media assetcan be accessed via its identifiercan be accessed via its identifier•Media identifer cant be = mediaMedia identifer cant be = medialocator !locator !TV SetLinkedTVServer• Video Analysis• Enrichment• Personalization• DeliveryBrowserCE-HTMLPlatform ClientExternalBroadcastHTTPBrowserVariantHbbTVVariantHTTPRepository• Annotations• Links toexternalResourcesEditorLoDCloudVideosfromScenariosBrowserHTML 5PC / TabletNetworkMaterialselection,AnnotationadjustmentLinkedTV Platform & REST APIhttp://data.linkedtv.eu
  • DerivingDeriving LinkedLinked MediaMedia•If different media assets share onlineIf different media assets share onlinedescriptions of their conceptualdescriptions of their conceptualrepresentations using Linked Data...representations using Linked Data...•Collecting media from differentCollecting media from differentsources which represent the samesources which represent the sameconcept is only the first stepconcept is only the first step•Link media together in differentLink media together in differentrelationships based on therelationships based on theconceptual relationship between theconceptual relationship between theconcepts they representconcepts they represent
  • Linked Media presentationLinked Media presentation•Relationships between media couldRelationships between media could(should?) be reflected in the presentation(should?) be reflected in the presentationlayer of the media applicationlayer of the media application•Spatial (map) and temporal (timeline)Spatial (map) and temporal (timeline)presentation is well knownpresentation is well known•WhichWhich conceptualconceptual relations could berelations could bevisualised and how?visualised and how?•Would it be intuitive to the mediaWould it be intuitive to the mediaapplication user?application user?
  • Where do we go from here?Where do we go from here?We need more online media descriptions which reflect Linked Media principlesWe need more online media descriptions which reflect Linked Media principles•Consider meaningfully distinct fragments of atomic media assetsConsider meaningfully distinct fragments of atomic media assets•Describe what a person perceives in the fragment using (Linked Data) conceptsDescribe what a person perceives in the fragment using (Linked Data) concepts•Use more descriptive properties to reflect how they represent a conceptUse more descriptive properties to reflect how they represent a concept•Publish the descriptions in a structured & semantic modelPublish the descriptions in a structured & semantic model•Follow Linked Data principles for applications to access and process media metadataFollow Linked Data principles for applications to access and process media metadataGenerate new links between online media based on related conceptual descriptionsGenerate new links between online media based on related conceptual descriptions•Come up with presentation ideas to visualise intuitively the links between mediaCome up with presentation ideas to visualise intuitively the links between media
  • LinksLinksLinkedTV Project websiteLinkedTV Project website http://www.linkedtv.euhttp://www.linkedtv.euMediaMixer Project communityMediaMixer Project community http://community.mediamixer.euhttp://community.mediamixer.euConnectME Annotation tool:ConnectME Annotation tool: http://annotator.connectme.athttp://annotator.connectme.atConnectME Hypervideo Player:ConnectME Hypervideo Player: http://player.connectme.athttp://player.connectme.atCMF (ConnectME Framework):CMF (ConnectME Framework): http://connectme.salzburgresearch.athttp://connectme.salzburgresearch.atLSI (Linked Services Infrastructure):LSI (Linked Services Infrastructure): http://production.sti2.org/lsihttp://production.sti2.org/lsi