The talk was delivered by Dorothea Tsatsou at the 7th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP 2012) from December 3-4, 2012 in Luxembourg, Luxembourg. More info: http://bit.ly/VN77sB

1. Television Linked To The Web
Dorothea Tsatsou, Vasileios Mezaris, Ioannis Kompatsiaris
Semantic personalisation in networked media:
determining the background knowledge
Centre of Research and Technology Hellas
Information Technologies Institute
SMAP 2012 • 3-4 December 2012 • Luxembourg www.linkedtv.eu

2. Towards networked media…
www.linkedtv.eu
Social TV
Second Screen content
push – breakthrough
with HTML5 mobile
http://www.designbynotion.com/metamirror-next-generation-tv/
The rise of
Smart TVs
Second screen apps show related
content without disturbing the TV view
LG SmartTV, pic courtesy http://www.wired.com/gadgetlab/2011/01/lg-smart-tv/
2 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

3. …to LinkedTV
www.linkedtv.eu
TV v ie w
Interweaving TV and Web
content into a single
experience
Se c o nd
s c re e n
Web enrichment of TV
content: a personal
activity that will not
disrupt TV viewing
3 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

4. Networked Media personalisation challenge
www.linkedtv.eu
Digital information overload
 Most crucial:
management of data
 Start: Background
knowledge
Digital information heterogeneity
4 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

5. Linked Open Data – Overview
www.linkedtv.eu
Current trend for content interpretation, therefore gaining popularity for
user preference representation
 DBPedia (different languages support)
 Broad coverage, shallow and inconsistent
 Freebase
 Google Knowledge Graph – content augmentation
 Community contributed
 YAGO
 schema.org
 …
 Domain-specific ontologies
 Geonames, MusicBrainz, …
5 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

6. Linked Open Data – Advantages
www.linkedtv.eu
 Broad coverage
 Structure over big data
 Community contribution,
 minimizing manual engineering needs
 evolving knowledge
 Interlinking of LOD vocabularies
6 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

7. Linked Open Data – Drawbacks
www.linkedtv.eu
 Inconsistency
 Shallowness
 Lacking user-pertinent information
User: Football, Manchester United
Content: Chelsea FC
LOD KB: Chelsea FC  Football 
User(Football fan)-pertinent KB:
disjoint Manchester United, Chelsea FC
 “Unnecessary” volume in knowledge itself and mappings
 Scalability
 Privacy
 Server-bound storage and processing
7 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

8. Formal Ontologies
www.linkedtv.eu
 General upper level ontologies
 DOLCE, SUMO, BFO, PROTON, …
 Advantages:
 Rich expressivity
 Consistency and well-defined structure
 Efficient alignment support over middle and domain ontologies for cross-domain
data handling
 Disadvantages:
 Generic
 Voluminous
 Ontologies for the media (TV) super-domain
 BBC Programmes ontology
 AVATAR ontology
8 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

9. User-specific ontologies & vocabularies
www.linkedtv.eu
 Cognitive Characteristics Ontology
 Upper characteristics schema (skills, interests, competences…)
 OCUM (Ontological Cognitive User Model)
 Imager, verbalizer, …
 GUMO
 Personality, facial expressions, motion, social environment, topics
 FOAF & FOAF-WI
 Relationships between people, social networking attributes, interest
expression schema
9 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

10. Aligning LOD and formal ontologies
www.linkedtv.eu
 Purpose:
 Unify relevant schemata under a more lightweight core ontology
 Unify individuals under a single type and mapping them to the uniform schema
 Individuals classification
 NERD ontology
 LOD mappings: Alchemy, DBPedia Spotlight, Extractiv, OpenCalais, Zemanta
 Schema alignment
 S-Match
 Class labels information
 AROMA
 Association rule mining
 BLOOMS
 Wikipedia & DBPedia for LOD schema alignment, can map to upper level formal ontologies
 BLOOMS+: contextual information to map LOD to PROTON
 LogMap
 Scalable automatic mapping & mappings consistency checking
10 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

11. Hypervideo content enrichment and linking
hypervideo to the web in LinkedTV www.linkedtv.eu
Cubism
Fauvism
Expressionism
CONTENT ENRICHMENT
11 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

12. The LinkedTV ontology (LUMO) design principles
www.linkedtv.eu
 Homogenization of multidisciplinary and diverse information
 Scalable modeling and matchmaking
 Context modeling
 Reuse of existing vocabularies and schemata
 Use only the knowledge that means something to the user and for the
user of a smart TV environment
 Expressive enough to make efficient inferencing
 Selective (automatic) domain specific knowledge incorporation
 Concept space based on more granular DBPedia subsets
 Mappings to classify content to the lightweight LUMO and not to
represent preferences per se
12 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

13. Towards LUMO: a snapshot
www.linkedtv.eu
13 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

14. Minimizing the concept space
www.linkedtv.eu
 Voluminous cross-domain information
 Multilingual content
Single conceptualisation
Avoid redundant steps in the
inferencing process
dbpedia:’Building’ ≡ de.dbpedia:Fußbal ≡
schema.org:’Civic Structure’ ≡ dbpedia:AssociationFootball ≡
lumo:Building lumo:Football
14 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

15. Mappings
www.linkedtv.eu
Annotation-indexed fuzzy mappings
Separate Mappings
<EquivalentClasses> To not only concepts
<Class abbreviatedIRI="dbpedia:LOCATION"/>
<Class abbreviatedIRI="schema:Place"/>
but possibly plain
<Class abbreviatedIRI="lumo:Location"/> keywords, labels
</EquivalentClasses>
15 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

16. Future work:
Knowledge pulling and learning www.linkedtv.eu
 Complexity minimization and scalability maximization upon
matchmaking:
 Contextual knowledge pulling
 Location and time-based context
 Current viewing context
 Evolving knowledge and group-specific knowledge
 User clustering and learning cluster-specific knowledge (rules)
 Expressivity to support rules
16 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

17. Conclusions
www.linkedtv.eu
 A more lightweight user-pertinent ontology: LUMO
 First version to be released by the end of the year under
data.linkedtv.eu/lumo
 Encompassing content related and context (user characteristics,
sensor-extracted data) related knowledge
 Emphasis on minimizing concept space yet maintaining sufficient
expressivity and meaningful information
 Multilinguality support
17 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas

18. Thank you!
www.linkedtv.eu
Any questions?
www.linkedtv.eu
www.iti.gr
18 SMAP 2012 • 3-4 December 2012 • Luxembourg Information Technologies Institute
Centre for Research and Technology Hellas