This document summarizes a presentation about the Centre for Telecommunications and Multimedia at INESC TEC. The center conducts research in two main lines: personalization to empower users with relevant data through context awareness and content recommendation, and collaborative annotation to obtain rich metadata by involving users. The goals are to address issues like information overload and lack of metadata by developing efficient and open solutions. Key application areas include TV/mobile multimedia consumption, tourism, assisted living, and remote work/monitoring. Milestones include prototypes for mobile context sensing and multi-view personalized experiences applied to domains like tourism and cultural heritage.

1. FromKnowledgeGenerationToScience-basedInnovation
Multimedia Technology Systems
Maria Teresa Andrade
Centre for Telecommunications and Multimedia - INESC TEC
INESC TEC Auditorium| April, 2015
Research and Technological Development | Technology Transfer and Valorisation | Advanced Training |
Consulting
Pre-incubation of Technology-based Companies

2. 14/4/15 Centre for Telecommunications and Multimedia
Representative Research Lines
• Personalisation
• empowering people with the right data at the right
time
• Collaborative Annotation
• involving users to obtain rich, meaningful metadata
and increase content re-usability

3. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Personalisation
!
• innovative solutions for personalised access and
consumption of multimedia content, through:
• context awareness: sensing and reasoning
• content recommendation
• content adaptation
• immersive multi-view experiences

4. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Personalisation

5. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Personalisation
Internet
focus of attention
multiview
server
add metadata
media
repository
(multiple
views +
metadata)
focus of
attention
media
repository
(multiple
views +
metadata)
video
stream
(view C)
2
focus of
attention
4
video
stream
(view R)
5
3
capture multiple views
request
content1
user
request
multiview cient
6

6. 14/4/15 Centre for Telecommunications and Multimedia
Scientific Motivation
• Heterogeneity and globalisation
• Information overload

7. 14/4/15 Centre for Telecommunications and Multimedia
Scientific Motivation
• Facts:
• proliferation of multimedia-enabled devices
• huge amount of multimedia content available online
• user-generated content adding to traditional content providers
• exponential rise of visual content consumption
• 300 hours of video are uploaded to YouTube every
minute
• mobile video traffic will increase 10 times in the next 4 years
• advances in video compression enable higher resolution
formats
• Consequences:
• users are becoming more demanding and want to access
content anytime, anywhere and anyhow
• still, they are overloaded with information

8. 14/4/15 Centre for Telecommunications and Multimedia
Scientific Motivation
• Information overload
!
!
!
!
!
!
!
!
!
!
!
• Let’s prevent this to happen!

9. 14/4/15 Centre for Telecommunications and Multimedia
Objectives
• Further investigate and devise efficient and open solutions
to improve initial results concerning:
• eye gaze based adaptive viewing
• degradation-free and minimal latency switching of views
• context acquisition and inference in mobile devices
• user profiling and clustering
• To look more seriously at aspects related to man-machine
interface and usability
• At a longer term,
• to develop analysis tools and metadata models to enable the
replication of human cognitive experiences across
temporally or spatially remote individuals

10. 14/4/15 Centre for Telecommunications and Multimedia
Milestones
• Robust prototypes for mobile context sensing and inference
(2015-2016)
• Operational prototypes for mobile context-aware
recommendation applied to tourism and to physical health
monitoring (2015-2017)
• Demonstrator for multi-view personalised experiences within
the cultural/tourism domain (2016-2018)
• Initial and consolidated prototypes for context-aware multi-
view remote monitoring of cultural heritage sites/
monuments (2017-2018, 2018-2020)
• metadata model and tools for sensing, representing and
reproducing multi-sensorial real-life experiences (2016-2020)

11. 14/4/15 Centre for Telecommunications and Multimedia
Application Areas
!
• TV consumption
• Mobile multimedia consumption
• Tourism
• Assisted living
• Remote (Collaborative) Work
• Remote monitoring
• of cultural heritage sites
• etc.

12. 14/4/15 Centre for Telecommunications and Multimedia
Application Areas

13. 14/4/15 Centre for Telecommunications and Multimedia
Application Areas

14. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Collaborative Annotation
!
• Based on concepts of crowdsourcing and Games With
a Purpose (GWAP) to motivate players
• users get involved through competition
• most popular and accurate tags are the ones
preserved
• use of validation and scoring mechanisms to
award players based on their success
• enabling to associate tags with time instants

15. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Collaborative Annotation

16. 14/4/15 Centre for Telecommunications and Multimedia
Research Line Collaborative Annotation

17. 14/4/15 Centre for Telecommunications and Multimedia
Scientific Motivation
• Content without metadata is as good as if it didn't exist
• Annotation is a time consuming task and
• when performed by humans has a high degree of
subjectiveness
• additionally, most users will not tag their content
• when performed by machines is not 100% reliable
• In most cases, video does not keep its semantic
characteristics throughout its entire duration
• Collaborative approach decreases costs and time required
for annotation
• and enables to retain useful and popular tags whilst
considering the temporal dimension

18. 14/4/15 Centre for Telecommunications and Multimedia
Objectives
!
• To further investigate the addressed topics to
improve:
• efficiency of search and content analysis
• accuracy of algorithms to validate inserted tags
• To research human-machine aspects aiming at
developing non-intrusive and appealing forms of
presenting content to users

19. 14/4/15 Centre for Telecommunications and Multimedia
Application Areas
!
• Video Archives
• Multimedia Content Management
• Metadata for Broadcasters
• Museums and cultural institutions
• Education

20. 14/4/15 Centre for Telecommunications and Multimedia
Milestones
!
• Benchmarking the proposed approach against
traditional annotation methods (2015-2016)
• Creation of a test bed with an external partner from
the media content industry (2015-2017)
• assessing the adequacy of proposed method with
real users
• Implementation of an operational prototype for a
concrete multimedia archive (2016-2018)

21. 14/4/15 Centre for Telecommunications and Multimedia
Partners
• Internal partners
– CTM, Information Processing and Pattern Recognition Area (2013 - …)
– audio, image and video analysis
– machine learning applied to A/V analysis
– CSIG (2014 - 2015 and beyond)
– Information Systems, GUI, GIS
– LIAAD (2013-2014 and beyond)
– data mining in social networks
• External partners
– Faculdade de Ciência e Tecnologia da UNL, Universidade de Lisboa,
IST, Loughborough University in London (LUiL), CNIT, CNR
– Publico, RTP, Cinemateca, Nonius

22. 14/4/15 Centre for Telecommunications and Multimedia
Equipment
• smart TV
• servers for running intensive processing tasks such as
A/V analysis, user profiling, data analysis for content
characterisation and context inference
• 360° video camera

23. 14/4/15 Centre for Telecommunications and Multimedia
Relationship with Tec4…
!
– Both research lines are aligned with Tec4Media
– Contributions in the form of
– identification of requirements and application areas
– exploratory work through MSc
– more advanced research through PhD
– elaboration of project proposals

24. 14/4/15 Centre for Telecommunications and Multimedia
Key Performance Indicators (until 2020)
• Publications
• Journals: 10
• MSc thesis: 15
• PhD: 3/4
• Organisation of international conferences: 5 (workshop in ACM
Multimedia; tentative workshop in IEEE ICME or IEEE ISM)
• Projects
– FCT: 2
– EU: 3/4
– Portugal 2020: 4/6
– Direct contracts: 2/4
– Other: MAT follow-up (ON2)

25. FromKnowledgeGenerationToScience-basedInnovation
Centre for Telecommunications and Multimedia - INESC TEC
INESC TEC Auditorium| April, 2015
Research and Technological Development | Technology Transfer and Valorisation | Advanced Training |
Consulting
Pre-incubation of Technology-based Companies
Multimedia Technology Systems
Thank you very much for your attention!
!25