LinkedTV is an EU funded project aiming to seamlessly interlink the TV and Web experiences. This presentation summarizes the results and achievements by the end of the project's second year, covering media analysis, annotation, linking, personalisation and interactive playout, based on two scenarios: Hyperlinked Documentary and Linked News. For more, visit www.linkedtv.eu

1. Television Linked To The Web
Web and TV seamlessly interlinked:
The LinkedTV project
Lyndon Nixon, scientific co-ordinator
MODUL University GmbH
lyndon.nixon@modul.ac.at
www.linkedtv.eu

2. The future of television
www.linkedtv.eu
 Bring the rich browsing experience people know
from the Web to television
 Fulfil the convergence of TV on the Web and Web
on the TV
 Make TV more instructional and informative by
uncovering the hidden knowledge in every video
frame
2
© 2013, the LinkedTV project

3. Pre-LinkedTV: Pop Up video!
www.linkedtv.eu
Viewers have always been
interested in additional
information about what they
are watching!
Pop Up Video is a VH1 television show that "pops up" bubbles — officially
called "info nuggets" — containing trivia, witticisms ... Most episodes of Pop
Up Video play four or five music videos each ... The bubbles that pop up in
each video generally appear about every 10–15 seconds ... Production
costs for each episode total about $30,000 (source: Wikipedia)
3
© 2013, the LinkedTV project

4. Trend: More TV over the Internet
www.linkedtv.eu
4
© 2013, the LinkedTV project

5. Trend: More Internet content on your TV
www.linkedtv.eu
Below: Boxee, hbbTV
5
© 2013, the LinkedTV project

6. Trend: More SmartTVs, more apps
www.linkedtv.eu
54 million SmartTVs
sold in 2012 will grow
to 170 million in 2017;
by then 64% of West
European households
will have at least 1
SmartTV.
„Smart-TV device forecasts“,
Informa Telecoms & Media
Samsung SmartTV app statistics
courtesy http://thenextweb.com/
gadgets/files/2011/05/5m-samsungapps-1.jpeg
6
© 2013, the LinkedTV project

7. But are Connected / Smart TVs smart?
www.linkedtv.eu
Consumers are using
complementary devices to surf
alongside TV, not the TV itself
Source: Yahoo Advertising Blog
http://www.yadvertisingblog.com
January 2011
Part of infographic courtesy http://socialitv.web.ua.pt
Full image at:
http://socialitv.web.ua.pt/index.php/contents/images/
7
© 2013, the LinkedTV project

8. New tech: Social TV
www.linkedtv.eu
 People tweeting about a program while they watch



Need agreement around a #hashtag
Shows interest in deeper interaction with the content
The Web generation are less interested in the passive TV experience
A shade under three-quarters of TV viewers with
broadband are surfing the web while they watch,
with 38 per cent of them discussing what they're
watching on social media. The proportion
discussing TV shows rises to 53 per cent if only
the 16 to 23-year-old demographic is included.
-- Ovum survey, published in TheRegister, 6
October 2011
8
© 2013, the LinkedTV project

9. New tech: Second screen apps
www.linkedtv.eu
 Second screen content push while watching TV



Relates to the program as an atomic object
Controlled by the content provider
Restricted to programs who have „opted in“
IntoNow or Shazam
identify what you are
watching and provide
associated content
Second screen
http://randymatheson.com/social-tv-platform-miso-launches-enhanced-experience-for-dexter-premiere/
apps show you related
content without disturbing the TV view
http://www.designbynotion.com/metamirror-next-generation-tv/
9
© 2013, the LinkedTV project

10. The LinkedTV future
www.linkedtv.eu
 There is a trend towards
„interactive images“ on the Web
 There is a trend towards
„interactive video“ (or „hypervideo“)
on the Web
ThingLink can
embed clickable
images in
Facebook
WireWax is
sharing clickable
video on YouTube
 We develop a new solution for
interactive media in a TV-integrated
environment: „LinkedTV“
„... a true TV ecosystem must functionally integrate the apps with the
television programming service and content being offered to the consumer.“
TVs can't be smart. Stop trying to make it happen. - WIRED Opinion, Gary Myer, Oct 2013
10
© 2013, the LinkedTV project

11. What makes LinkedTV special?
www.linkedtv.eu
 Build upon Web and broadcast specifications and standards


Make sure LinkedTV is part of the whole industry, not another proprietary
fragment
HTML5, Linked Data, W3C Media, hbbTV, Open IPTV Forum
 Reuse the rich wealth of data and services on the open Web


Give added value at low cost
Extendible to closed data and services for specialised offers
 Automate the annotation and enrichment as far as possible


Broadcasters barrier to entry is the complexity and effort to make hypervideo
Current approaches will never scale to the amount of video being produced
11
© 2013, the LinkedTV project

12. Scenarios
www.linkedtv.eu
Scenario 1:
Interactive News
Show
 Professional news
content produced
by RBB
 Seed content: local
news show "rbb
Aktuell"
Scenario 2:
Hyperlinked Documentary
 Cultural heritage
content from S&V
 Seed content:
"Tussen Kunst &
Kitsch" from AVRO
(Antiques
Roadshow format)
Scenario 3:
Media Arts
 MEDIADROM
competition for
defining new ideas
for linked television
 Concepts will be
promoted at Mons:
European Capital of
Culture 2015.
12
12
© 2013, the LinkedTV project

13. User trials for LinkedTV usage insight
www.linkedtv.eu
Scenario 1:
Scenario 2:
Interactive News Show Hyperlinked
Documentary
 News is often
watched together and  Tussen Kunst &
over a meal!
Kitsch viewers do
 Viewers have more
want to look up
complex information
information
needs when watching  Objects and their
news
history of primary
 Enriched news
interest
viewing could be
 Bookmark for later,
parallel to the main
not during the show
news, without
additional interaction
13
© 2013, the LinkedTV project
Also: industry insight
on consumption
trends
 Second screen and
social media
integration are the
key trends
 News and
documentary are the
most interesting
formats for
enrichment
 Additional info
should be compact,
bookmarkable
13

14. Scenario for a hyperlinked documentary
www.linkedtv.eu
See the demo: http://www.linkedtv.eu/demos/hyperlinkeddocu
Content courtesy
14
© 2013, the LinkedTV project

15. Scenario for an interactive news show
www.linkedtv.eu
See the demo: http://www.linkedtv.eu/demos/linkednews
Content courtesy
15
© 2013, the LinkedTV project

16. How will it work?
www.linkedtv.eu
Video object and
text detection
Examples of Paintings by Jan Sluijters
… and their digital images
Connection to
concepts
Expansion of
related concepts
Linking to
related content
“...schilderij van Jan Sluijters....”
dbpedia.org/resource/Jan_Sluyters
16
linkedtv.eu
© 2013, the LinkedTV project
Personalisation
engine

17. Concept Detection
www.linkedtv.eu
Video object and
text detection
LinkedTV works on services using &
combining different forms of audiovisual
content analysis:
– Face detection
– Textual analysis (subtitles,
transcripts)
– Keyword extraction
– Multilingual automatic speech
recognition
– Speaker identification
– Concept detection in video
– Object re-detection
– Shot segmentation
This provides us with identification of
“...schilderij van Jan Sluijters....”
meaningful media fragments and
summaries of their content analysis
17
linkedtv.eu
© 2013, the LinkedTV project

18. Concept Connections
www.linkedtv.eu
Connection to
concepts
Keywords and text fragments can be connected to
uniquely identified, distinct named entities
using online entity extraction services (NERD,
http://nerd.eurecom.fr)
– Media annotations are represented in RDF
and stored in LinkedTV platform
– Re-use of existing Web vocabularies
e.g. schema.org, NERD, LSCOM, DBpedia
– Named Entity Recognition uses
statistical & knowledge-based approaches
(aiming at entity meaning disambiguation)
– Extended by complementary visual
analyses (e.g. a persons name in text with
a face detected in a video fragment)
dbpedia.org/resource/Jan_Sluyters
18
linkedtv.eu
© 2013, the LinkedTV project

19. Concept Expansion
www.linkedtv.eu
Paintings by Jan Sluijters
Expansion of
related concepts
19
The use of Linked Data in the identification
of concepts in LinkedTV means we can
expand concepts along different facets, i.e.
allow users to explore in terms of their
different interests in a given concept.
For example, for an artist like Jan Sluijters,
LinkedTV can link into:
 Additional information, e.g.
 biography of artist
Leo Gestel
has art style
 style of painting
 Related information, e.g.
luminism
has art style
has art style
 artists from same
Piet Mondriaan
Jan Sluijters
period
 paintings in similar style
 related styles
linkedtv.eu
© 2013, the LinkedTV project

20. Linking to related content
www.linkedtv.eu
… and their digital images
Linking to
related content
20
LinkedTV provides enrichment services
which provide recommendations for
related online content (Web pages,
images, audio, video) pertaining to the
concepts in the TV program:
– Fresh Social Web content coming
from e.g. Twitter and Facebook
– User Generated content coming
from e.g. Flickr and YouTube
– Whitelist content coming from
partner Websites like public
broadcasters in Germany or cultural
heritage archives in the Netherlands
– Extendable and configurable by
source and media type
linkedtv.eu
© 2013, the LinkedTV project

21. Personalisation engine
www.linkedtv.eu
Personalisation
engine
21
The LinkedTV Platform pushes annotation and
enrichment information to the client (LinkedTV
Player)
– Both annotations and enrichments include
further metadata about the topics they relate
to, e.g. Jan Sluijters + Amsterdam, BER
Airport + Berlin
– A Personalisation engine on the Player
filters & ranks the annotations and
enrichments according to a locally stored
user model, e.g. interested in Amsterdam vs
interested in Berlin
– The user model can be both manually edited
online (LUME) or learnt by the LinkedTV
system (attention tracker, event tracking)
– All personalisation is opt in and private,
respecting ethical issues
linkedtv.eu
© 2013, the LinkedTV project

22. Packaged into a LinkedTV Platform
www.linkedtv.eu
The LinkedTV Platform
encapsulates the
functionality of the
previous 4 steps.
The main core of the
platform is an
Application Service Bus
which monitors and
triggers events across
the components,
realising the end to end
workflow from a video
ingestion to enriched
playout.
Communication between platform components and between platform and
client is realised using RESTful APIs.
22
© 2013, the LinkedTV project

23. Playout on the LinkedTV Player
www.linkedtv.eu
LinkedTV enriched video playout needs a UI for Linked Television
 The Springfield multiscreen platform abstracts the video and its
enrichments from the device where they are displayed
 Viewers can browse the topics of the program
along „Who“, „Where“, „What“ axes
 Enrichments can be browsed to, or shared,
or bookmarked for later
23
© 2013, the LinkedTV project

24. Future LinkedTV Player UI
www.linkedtv.eu
LinkedTV player UI will evolve with the Platform technology
 Richer displays of associated information
 HbbTV implementation
 Gesture control
24
© 2013, the LinkedTV project

25. Follow LinkedTV & keep up with
the future of television
Web: www.linkedtv.eu
Twitter: @linkedtv
25
© 2013, the LinkedTV project
www.linkedtv.eu

26. www.linkedtv.eu
26
linkedtv.eu
© 2013, the LinkedTV project