Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
ConnectME: connecting content
for future online TV & video
Lyndon Nixon
lyndon.nixon@modul.ac.at
Last update: September 25...
Connected Media Experiences
• New services deriving added value for video by
combining it with complementary content from
...
Growth of Internet on the TV

Below: Boxee, ConnectedTV, Samsung Smart TV (http://thenextweb.com/gadgets/files/2011/05/5ms...
Growth of TV on the Internet
Growth of mobile video
Video streaming accounts
for 37% of all mobile traffic
Of all video streaming
traffic, YouTube acco...
Trending towards connected
media
Associating content to video

http://notube.open.ac.uk/sugartub

http://webmademovies.etherworks.ca/popcorndemo/
Emerging hypermedia

Asterpix‘s Asterbot
discontinued!
Cf. Videoclix, Overlay.TV, Klickable, ClickVideo

Wirewax
http://ww...
What could be a
connected media experience?
Where is Obertauern?

Which peak is that
Can I ski there? Ho

What is the buil...
How to experience connected media?
The business case for hypervideo
• Offering an added value service
– Solely to win more visitors & keep them using the
ser...
The technical case for
connected media
• Channel annotation, enrichment and delivery
are all services of the ConnectME fra...
Content workflow
• Upload video in partner CMS
• Fill in basic information and „commit“ video to
the ConnectME Framework (...
Relationship Linked media framework (LMF) <->
Connected media framework (CMF)
• CMF wraps the LMF reusing its inherent med...
Channel annotation
• Grouping of videos into a „channel“
• Easy to use tools for content owners or hosts
to annotate video...
Yoovis use case: ConnectME
channel

Courtesy Yoovis GmbH, www.yoovis.tv
Integration of annotation
Pre-annotation in Yoovis use case
• Yoovis CMS calls a Web service on CMF which
accepts the mediaRSS format
– „registers“ ...
Pre-annotation in the PS Media
use case
• PS Media CMS pushes to CMF a RDF
description generated in Drupal
– RDF Module ma...
Pre-annotation (generic)
• Any client could directly call a Web service on
CMF which takes the parameters title,
descripti...
Annotation suite
- HTML 5, CSS 3,
responsive layout, based
on Twitter Bootstrap
- jQuery, JS based UI
- Timeline based int...
Annotation suite
- Annotations are RDF and
concepts are using LOD
identifiers
- Annotation export in N3,
RDF/XML and Turtl...
Annotation model

published at www.connectme.at/ontology
Re-uses a subset of W3C
Media Ontology and
Open Annotation Model
...
ConnectME annotation instance
<
http://connectme.at/yoovis_video1.ogv
> a ma:MediaResource;
ma:hasSubtitling <
http://conn...
Annotation suite
Production annotation suite
http://annotator.connectme.at
Annotation via the CMF
• Video metadata is turned into ConnectME
annotations on the CMF
– Ingestion services generate top ...
Enrichment via the CMF
• Each concept needs to be linked to related
content items
– The player widgets could query this di...
Direct & indirect links
Direct links in the concept metadata to related
content can be transformed to the common
model by ...
Linked Services Infrastructure
• http://production.sti2.org/lsi
– Web interface and REST API
– Input: a LOD URI, output: R...
Linked Services Infrastructure
Middleware to different Web APIs (YouTube, Flickr, Ookaboo, Foursquare
Venues, Instagram)
»...
Linked Services Infrastructure
RDF response format based on
W3C Media Ontology
Types are e.g. Image, VideoTrack
Locator po...
Channel enrichment
• ConnectME player with optional styling
(customer branding)
• Pre-selection of content sources and
pre...
Hypervideo playout & browsing
Based on open source
LIME player project
HTML5 & CSS3 based
Video format support as
per brow...
Player implementations
• Desktop
– Administrative & user interface
– Widgets as extra webpage
regions („standard“ player)
...
Player widget model
Standard widget model
is tiles-based* and abstracts the
Interaction model from
• Desktop: point and cl...
„Core“ player widgets
• Info (DBPedia)
• Media (LSI)
• Map
„Extended“ player widgets
Extended widget model
needs definition of:
- Annotation trigger (concept type)
- Content to disp...
„Live TV“ player widgets
PS Media use case
defines these additional widgets
- Actor
- Character
- Director
Demos
Production workflow has demos of ConnectME,
at http://player.connectme.at
IPTV example:
Choose „Fahrenheit 9/11“ and...
Upcoming SlideShare
Loading in …5
×

ConnectME: connecting content for future TV & video

1,096 views

Published on

Today, media material is increasingly digital and shifting to delivery via IP and the Web,
including cultural artifacts or broadcast television. This opens up the possibility of new
services deriving added value from such material by combining it with other material
elsewhere on the Web which is related to it or enhances it in a meaningful way, to the
benefit of the owner of the original content, the providers of the content enhancing it and
the end consumer who can access and interact with these new services. Since the
services are built around providing new experiences through connecting different related
media together, we consider such services to be Connected Media Experiences
(ConnectME).

In particular, much rich depth of information and service functionality associated to
content in video is not derived today due to a lack of suitably granular description of
video, including linking of video objects to the concepts they represent. For example,
news reports about local tourism and events are not linked to tourist information and
event–related services which a viewer may (spontaneously) wish to access and make
use of.

The technological result of the ConnectME project will be an end-to-end service platform
to host those added-value services over different networks, providing the common
required functionality of each service: multimedia annotation and subsequent enrichment
with related content from the Web, combined with the packaging and delivery of
synchronized multimedia presentations to the end device. At the device, intuitive user
interfaces must be developed so that the selection of on-screen objects and the
browsing of the associated content can be done in a non-disruptive and intuitive fashion.
As a result, ConnectME facilitates a new interactive media experience built on top of the
convergence of TV/video and the Web.

  • Be the first to comment

  • Be the first to like this

ConnectME: connecting content for future TV & video

  1. 1. ConnectME: connecting content for future online TV & video Lyndon Nixon lyndon.nixon@modul.ac.at Last update: September 25,2013 More at www.connectme.at Contact : info@connectme.at
  2. 2. Connected Media Experiences • New services deriving added value for video by combining it with complementary content from the Web • Linking of video objects to the concepts they represent, and hence related content about those concepts • Provisioned as an enhanced in-video experience • Delivered via an end-to-end service platform hosting the added value services over different networks
  3. 3. Growth of Internet on the TV Below: Boxee, ConnectedTV, Samsung Smart TV (http://thenextweb.com/gadgets/files/2011/05/5msamsung-apps-1.jpeg)
  4. 4. Growth of TV on the Internet
  5. 5. Growth of mobile video Video streaming accounts for 37% of all mobile traffic Of all video streaming traffic, YouTube accounts for 45% A Cisco study* on mobile traffic growth expects • 66% of all traffic by 2014 will be video • having increased 66-fold from 2009 to 2014 * Cisco‘s Visual Networking Index Global Mobile Data Forecast 2010-2015
  6. 6. Trending towards connected media
  7. 7. Associating content to video http://notube.open.ac.uk/sugartub http://webmademovies.etherworks.ca/popcorndemo/
  8. 8. Emerging hypermedia Asterpix‘s Asterbot discontinued! Cf. Videoclix, Overlay.TV, Klickable, ClickVideo Wirewax http://www.wirewax.com
  9. 9. What could be a connected media experience? Where is Obertauern? Which peak is that Can I ski there? Ho What is the buildin If it is a hotel, how the accommodat
  10. 10. How to experience connected media?
  11. 11. The business case for hypervideo • Offering an added value service – Solely to win more visitors & keep them using the service – Encourage upgrades to premium packages (by content provider, by end consumer) • Monetarization of the links – E-commerce (drive traffic to electronic purchases) – Content partnerships (drive traffic to others‘ content) – Ad inserts
  12. 12. The technical case for connected media • Channel annotation, enrichment and delivery are all services of the ConnectME framework
  13. 13. Content workflow • Upload video in partner CMS • Fill in basic information and „commit“ video to the ConnectME Framework (CMF) • CMF includes video instance and preannotates it from the basic information and video analysis components • Annotation tool provides preview of preannotation and means to finalise annotation • CMF handles links to content enrichments • Preview in ConnectME player with widgets rendering different content enrichments
  14. 14. Relationship Linked media framework (LMF) <-> Connected media framework (CMF) • CMF wraps the LMF reusing its inherent media & metadata handling functionality and realising the ConnectME workflow on top. Analysis, annotation Visual analysis LMF Speech to text Workflow Linked Media Store Concept extraction REGISTRATION Interlinking, enrichment Annotation views ConnectME ontology based Linked Media engine rul es LOD REST Annotation tool Interlinked media generation Presentation engine Enrichment playout Presentation, playout REST AUTENTICATION Hypervideo browser
  15. 15. Channel annotation • Grouping of videos into a „channel“ • Easy to use tools for content owners or hosts to annotate video segments with links to semantic concepts – Manual annotation tool – Supported by • automatic classifiers over visual features, audio, subtitling/transcripts, or other annotations incl. crowdsourced tagging • tool tips for choosing concepts from pre-selected sources
  16. 16. Yoovis use case: ConnectME channel Courtesy Yoovis GmbH, www.yoovis.tv
  17. 17. Integration of annotation
  18. 18. Pre-annotation in Yoovis use case • Yoovis CMS calls a Web service on CMF which accepts the mediaRSS format – „registers“ a video instance in the CMF – Includes video metadata directly extractable from the mediaRSS, e.g. locator(s) – Generates video annotations from title, description and tags from the author • Attachs these „pre-annotations“ as 'dct:subjects' in the video metadata with local Yoovis terms, mapped where possible to LOD such as DBPedia instances
  19. 19. Pre-annotation in the PS Media use case • PS Media CMS pushes to CMF a RDF description generated in Drupal – RDF Module maps Drupal node data to RDF – LMF Module pushes this RDF into our server – Generates video annotations from title, description and subtitles from the EPG • Attachs the „pre-annotations“ as in Yoovis use case • Entity extraction from subtitles can associate entities already to a temporal segment (store as annotation)
  20. 20. Pre-annotation (generic) • Any client could directly call a Web service on CMF which takes the parameters title, description, URL & subtitles – „registers“ a video instance in the CMF – Includes locator(s) from URL parameter – Generates video annotations from title, description and subtitles • Attachs the „pre-annotations“ as in Yoovis use case • Entity extraction from subtitles can associate entities already to a temporal segment (store as annotation)
  21. 21. Annotation suite - HTML 5, CSS 3, responsive layout, based on Twitter Bootstrap - jQuery, JS based UI - Timeline based interface - Allows adding/editing/ deleting annotations in/to the CMF - Allows registering videos to and loading video annotations from the CMF - Multilanguage support (currently EN, DE) - Access via OpenID
  22. 22. Annotation suite - Annotations are RDF and concepts are using LOD identifiers - Annotation export in N3, RDF/XML and Turtle - Manual annotation with LOD concepts (URIs) and HTML content (URLs) - Currently Plugins for dbpedia and geonames search APIs - Plugin based architecture allows adding further LOD sources easily
  23. 23. Annotation model published at www.connectme.at/ontology Re-uses a subset of W3C Media Ontology and Open Annotation Model Extends mao:hasKeyword to subproperties of explicitly/implicitly seeing/mentioning concepts. Pre-annotations attached to video using dct:subject and skos:Concepts cmo:preferredLabel overrides concept labels cmo:hasContent points to static Web content
  24. 24. ConnectME annotation instance < http://connectme.at/yoovis_video1.ogv > a ma:MediaResource; ma:hasSubtitling < http://connectme.at/yoovis_video1.srt >; ma:hasLanguage [ rdfs:label "de" ] ; ma:hasContributor < http://www.yoovis.at>; ma:creator http:// www.yoovis.at/imprint#me ma:date "2011-09-13"^^xsd:date; ma:hasRelatedLocation "27.14, 16.83"^^geo:point; ma:isCopyrightedBy < http://www.yoovis.at> ; ma:hasPolicy < http://creativecommons.org/licenses/ ...>; ma:hasFragment http:// connectme.at/fragment/1. http://connectme.at/fragment/1 a ma:MediaFragment; ma:locator < http://connectme.at/yoovis_video1.ogv#t= >; cma:explicitlyMentions < http://dbpedia.org/resource/AntiBaby%20P >. <http://connectme.at/annotation/1> a oac:Annotation; oac:target < http://connectme.at/fragment/1>; oac:body < http://dbpedia.org/resource/AntiBaby%20P >; dcterms:creator http:// open.id/person1; dcterms:created "2013-0219T16:20:38+00:00"^^xsd:dateTime .
  25. 25. Annotation suite Production annotation suite http://annotator.connectme.at
  26. 26. Annotation via the CMF • Video metadata is turned into ConnectME annotations on the CMF – Ingestion services generate top level metadata such as video title, description, author, license... • Analysis of textual metadata to extract keywords a task of the Apache Stanbol Enhancer – A generic enrichment chain for entities->DBPedia – Specific enrichment chains for Yoovis and PS Media (entities->keywords(->LOD))
  27. 27. Enrichment via the CMF • Each concept needs to be linked to related content items – The player widgets could query this directly but – The CMF can cache relevant links so that the player can make simple queries for content • Common data model for links regardless of source, type etc. – See the Media Ontology for Linked Media
  28. 28. Direct & indirect links Direct links in the concept metadata to related content can be transformed to the common model by extending CMF reasoning rules („Annotation views“) ($c http://xmlns.com/foaf/0.1/depiction $m) -> ($m http://connectme.at/ontology#explicitlyShows $c) - Indirect links must be derived for types of concept using other Web based content sources (APIs)
  29. 29. Linked Services Infrastructure • http://production.sti2.org/lsi – Web interface and REST API – Input: a LOD URI, output: RDF for matching media resources
  30. 30. Linked Services Infrastructure Middleware to different Web APIs (YouTube, Flickr, Ookaboo, Foursquare Venues, Instagram) » „lowers“ the input concept to query parameters such as label, long/lat ... » „lifts“ the API response to RDF links for the CMF, where a content item is described and connected to the original concept by ma:hasKeyword LMF API query LSI module RDF answer Linked Media Store Linked Services Infrastructure New module To-beinterpreted answer (wrapped) query „lifting“ and „lowering“ handled by LSI Some online content source
  31. 31. Linked Services Infrastructure RDF response format based on W3C Media Ontology Types are e.g. Image, VideoTrack Locator points to the actual resource URL hasKeyword links the resource to LOD URIs related to the resource We can include further metadata as provided from the Web API (e.g. when uploaded, no of views, duration, size...)
  32. 32. Channel enrichment • ConnectME player with optional styling (customer branding) • Pre-selection of content sources and presentation (widgets) by administrator (administration interface) • On-the-fly video enrichment by selected widgets at play-out (user interface) • Administrator access to player analytics – playback, click-through, in-widget actions
  33. 33. Hypervideo playout & browsing Based on open source LIME player project HTML5 & CSS3 based Video format support as per browser support - MPEG-4 (IE, Safari, Android) - WebM and OGV (Firefox, Chrome, Android, Opera) Production player http://player.connectme.at Development player http://tkurz.github.io/lime Software project http://github.com/tkurz/lime (Apache 2.0 license)
  34. 34. Player implementations • Desktop – Administrative & user interface – Widgets as extra webpage regions („standard“ player) – Fullscreen (with widgets as overlays) • TV – GoogleTV (fullscreen, no video pause, supports remote control) • Mobile – Tablet version (fullscreen adapted to touch, min 7“ screen) „The“ player incorporates both standard and fullscreen impl.
  35. 35. Player widget model Standard widget model is tiles-based* and abstracts the Interaction model from • Desktop: point and click • Mobile: touch • TV: remote control * A tile is 300x300px and a typical widget is max 600x600px
  36. 36. „Core“ player widgets • Info (DBPedia) • Media (LSI) • Map
  37. 37. „Extended“ player widgets Extended widget model needs definition of: - Annotation trigger (concept type) - Content to display (query CMF) - How to display it (tiling)
  38. 38. „Live TV“ player widgets PS Media use case defines these additional widgets - Actor - Character - Director
  39. 39. Demos Production workflow has demos of ConnectME, at http://player.connectme.at IPTV example: Choose „Fahrenheit 9/11“ and TV player Mobile tourism example: Choose „Schladming“ and tablet player

×