Anatomy of a Cross Media Best Practice Network for Media Aggregation and Fruition

8th International Conference on Signal Image Technology & Internet Systems, 2012

Paolo Nesi
University of Florence, Department of Systems and Informatics, DSI
DISIT-Lab, Distributed Systems and Internet Technologies
Via S. Marta 3, 50139, Firenze, Italy
Email: paolo.nesi@unifi.it, www: http://www.disit.dsi.unifi.it
TEL: +39-055-4796523, 567,
FAX: +39-055-4796363, +39-055-4796469
Keynote at SITIS 2012 Conference, 26/11/2012 1

 Objective and overview
 Networking & Tools
 Content & Tools
 ECLAP Architecture

2
Keynote at SITIS 2012 Conference, 26/11/2012

3

Archive
partner
Archive Original Content/Portals
Library partner
Archive
Library partner
partner
Library Search/Query
partner

Content
ECLAP
Social Agg. Content
Performing
Service Metadata art
Portal Services Institutions,

Social Srv. students,

Search and browsing metadata lovers


ECLAP

5

 Content aggregators to
 Aggregate and Enrich content
 prepare content for Europeana, and
 distribute content to ECLAP users
 Working Groups on best practice
reports about tools for performing
art content for:
 Education and Training
 Intellectual Property and business
models
 Digital libraries and archiving


Automated -PC, MACos, linux, …
Back office -iPhone, iPod,
Windows Mobile, ….…

ANY content

Agg. Content
ANY content
Content
Content
archive
Content Services
Library archive
Content
Library archive
partner
Library
partner

8

 >30 partner from Europe + Chile, Russia, South Africa
 >100.000 content, 300.000 items
 Descriptive metadata in 13 languages
 Interface in 21 languages
 Accessed from 184 Countries
 Mainly from: Italy, UK, Spain, US, NL, B, Sl, Hu, Fr, Pl,
Pt, Gr, D, Bz, Ind, Sw, Au, Rs, Ca, etc.
 More than 35 groups
 More than 500 different file formats

9

 Content & Tools

10

 35 groups
 User groups, discussion, forums, mailing
lists
 Connection with social network for
share: Authentication, social icons
 Contact gathering
 Suggestions and recommendations to
ECLAP users
 Events setup and management
 Mailing newsletters, notifications,
suggestions ….
 Custom look and feel with group
designer
 Embedding content on third party
 …


12

13

Recipient of the suggestions
User Content Group
(played by a user) (leader or members)
Users Proposing to a user Proposing at a group
possible colleagues / --no sense-- responsible possible
friends interested colleagues to be
Suggested elements

D(U(s,d);U(s,d)) invited
Contents Proposing to a user Proposing at a play of a Proposing at a group
possible interesting content similar content members possible interesting
contents items content
D(C(s);C(s)) (not much different with
respect to C-C combination)
Groups Proposing to a user Proposing at a play of a
possible interesting content possible interesting --no sense--
groups groups in which similar
contents are discussed
Ads Proposing to a user Proposing at a play of a Proposing at a/all group
possible interesting ads content the possible member/s possible interesting
interesting ads ads

14

 Why ?
They have the similar

age, lang, localtion, ..
 Access/play similar
content
 Provide similar content
 Have similar interests
 Have similar friends
 …
 Different SNs present
different users’ intentions

15

suggestion , , ,

 ,
, , , , , , , , , , ,

, , ∙ ∙ , ∙ ∙
, ∙ ∙ , ∙ ∙
, ∙ ∙ , ∙ ∙
, ∙ ∙ (A,B)

Hyp:
• Acceptance of Static based suggestions: age, location, language,. …
• Acceptance of Dynamic Suggestions based on similarity of interests
• Acceptance of Random Suggestions
• Acceptance of Strategic Suggestions based on # of connections….

16

5
incidence
4 3,2 regression
3
2 1,6 R multiple 0, 9624
1,3
1 0,1 0,3 F - Value 131,7795
0 Sign of F 2,3389E-33
-1 -0,4
played similar age language locality groups
content interests

Random selection Strategic
 Number of friends

Deduction of (i) the mostly accepted suggestion kinds
and of (ii) the best weighting factors

17

 Content & Tools

18

 Cross media content, any content kind
 Metadata model, Metadata Editor
 Multilingual Indexing and Search, Faceted, multilingual
 Recommendations
 Content Aggregation: playlists, collections, courses
 AudioVisual relationships among content MyStoryPlayer

 Automated and Scalable content management

 Content IPR modeling and CAS

19

 Informative Content • Performance
 Video, audio, images, • Master classes comments
documents • Scene Sketches
 3D, animations, Braille • Scenography rating
• Scenes
 Slide, Video-Slide, courses
• Private lives of
 eBook, ePub, Mpeg21, relationships
artists
intelligent • Scores
 Aggregated Content: • Braille technical
 Playlist, Collections • BackStage Stills
 Annotations, Synchronization • Choreography Dynamic
• Morals
 Support and networking
• Poster recommend
content:
• Booklets
 Blog, WebPage, Events,
• Magazines Music
comments, ……………
• Audio ballets
forum, votes, messages, …

20

 Describe the ECLAP content
 Dublin Core: general, multilingual and multi-instances
(e.g., title, authors, description, subjects, ..)
 Semantic: content description, taxonomy, tags,
annotations, etc., multilingual & multi-instances
 Groups: one or more groups
 Technical: duration, file format, dimension, editor,
publisher, any IDs, etc. etc.
 Augmented Reality: GPS, QR,..
 Intellectual Property: IPR, licences for MD, licences for
profiling the conditional access
 Workflow metadata: status, dates, versioning, etc.
 Etc.
21

Group/Channel
0..n
0..n

0..n 0..n
0..n 0..n 1 0..n
TaxonomyTerm Content Comment Performing
Arts

Metadata Dublin Core

Technical
1..n
Blog WebPage Forum Object

0..n

Playlist Document Collection
0..n 1..2 0..n
Annotation AVObject 1..n

Image Video Audio
26

recommend
haveColleague

registeredTo
RegisteredUser Group/Channel
0..1
0..n
preferredTerm
Action 0..n 0..n
0..n 0..n 1 0..n
TaxonomyTerm Content Comment
ToRate

ToComment
performedOn

ToTag

ToSocialShare
Page Forum Object
ToRecommend

ToEnrich

ToValidate

ToPublish

27

 Indexing & Search system
 Based on Apache Solr
 Multilingual aspects
 Translate the metadata or translate the query?
 We use metadata translation
 Indexing schema
 Dublin Core + DCTerms (multi language)
 Performing Arts
 Technical (provider, content type, GPS, IPR, duration, quality, …)
 Groups associations (multi language)
 Taxonomy associations (multi language)
 Comments & multi language tags
 FullText of the textual digital resources

28

 Y: only one metadata set;
 Y/N: metadata set not
complete
 T: only title of the metadata
set,
 Yn: yes with n possible
languages (i.e., n metadata
sets);
 Ym: m different comments
can be provided, each of
them in a specific
language.

29

 Search for:
“dario fo”,
Video
 394 results
 Of which:
 ….
 Combining the
filtering
 51 results
 Of which…

31

32

 Optimisation Obtained by using Simulated Annealing
 w1 = 68:4739, w2 =31:7873, w3 = 0:2459, w4 = 9:8633,
 w5 = 13:2306, w6 = 2:1720, w7 = 3:9720
 MAP, Mean Average Precision = 0,82
33

# Full Text # Faceted # Last #Featured # Popular
users Query Query Posted List List List
simple 323 24 4 22 17
registered

partners 1094 21 27 19 9

anonymous 2634 147 234 302 213
Total 4051 192 265 343 239
Clicks after 1564 200 318 2799 231
query/list

34

 Click order distribution

First page

35

 Similarity by
taxonomical
classification
 Hierarchical clustering
solutions
 Similarity by Text, MD
 Frequency of
keywords
 Similarity by Content,
CBIR
 Several algorithms
 Next Steps
 combinations

36

http://www.eclap.eu/cbir
37
SCP course, Univ. Florence, 2012

 Play Lists
 Collections
essay
 Courses

 Without
resource
change

 Diff. models
 Diff. Semantic
to play them
 MD as content
38

 Playlist
 Sequence of segments of videos and/or audio and
images kept for a time duration
 They are show/played in sequence (the images have an
associated duration imposed by the creator)
 Collection
 A set of content (audio/video/images, docs, playlist,
animations, 3D, etc., any kind)
 They can be downloaded from mobile content organizer
 Playlists/Collections have their specific metadata which
are indexed as regular content

39

 Use Cases:
 Multiple views of the same action
 Comment audiovisual segment with others
audiovisual segments: comparison, scene
analysis, etc.
 Formalize relationships among media
 RDF model
 RDF progressive player

40

V1
s1 e1
Case a) V2
 One2One s2 e2

V1
s1 e1
Case b) V2
s2 e2
 Synchronization
V1
 Sequences s1 e1
Case c) V2
s2 e2

V1
 Explosive Formalized as
V2
S1=e1

s2 e2

V1 V2 V1
Executed as s2 e2
S1=e1
41

42

 User may
 Navigate on the non linear relationships defined by the
several audio visual annotations
 Record/play its experience in navigating on the non
linear relationships defined by the several annotations
 Share the Experiences to other users
 Perform semantic queries

 User may annotate the audiovisual content with
 Audio Visual annotations, taking segments
 Textual annotations
 Semantic annotations/relationships
43

 http://www.eclap.eu/drupal/?q=en-
US/msp&axoid=urn%3Aaxmedis%3A00000%3Aobj%3A04e0caef-b33b-4f4a-
ba50-a80d96766192&axMd=1&axHd=1

45

b)

a)

c)

d)

46

Explosive

47

)

48

http://www.eclap.eu/d3/graph.html http://www.eclap.eu/d3/graph2.html

49

50

One2One Is-a Label
Explosion
relationship
Synchronization description

Sequential has relation/
is related of ECLAP
has MediaReference/
Content
is MediaReference of
refTo Is-a
Segment Media ECLAP
Is-a classification

start end Audio Video Image

51

 Content & Tools

52

O
AXCP back office services ECLAP Social
ECLAP A
Metadata I
Ingestion and Harvesting
Service Portal
Networking
Ingestion P
M
Server H Resource Injection
IPR Wizard/CAS
Content Analysis
Archive Metadata Editor
Content
partner
Archive Retrieval Content Processing
Library partner
Archive
Library partner Content Aggregation and Play
partner
Library
Semantic Computing and Sugg.
partner
Content Indexing and Search

Content Upload Management
Metadata

Metadata
Database +
Export
semantic database Content Upload

Social Network
connections

E-Learning
Support


 Based on AXCP Semantic Media Grid (IEEE Multimedia, April 2012)
 More than 15.000 files per day in the present architecture with 15 server
 To automate:
 Content ingestion from more than 35 archives/portals, metadata and
content crawling, massive content ingestion
 Metadata transcoding and enrichment: from any format to ECLAP
ingestion format, saving and mapping all metadata
 Content indexing: more than 500 metadata, 13 languages, cross media
content, algorithms for CBIR
 Content adaptation: for PC, Mobile, etc.  over than 500 media types
 Suggestions and recommendations production: by text, by images, by
profiles; UU, CU, CC, ..
 Production of profiled newsletters
 Assessment of metadata quality
 Publication of metadata towards Europeana
 Export to other Social Networks
 Etc.
54

Local:
 Content collection
 Search/query
 Navigations..
taxonomy
 Suggestions
 …


AXCP backoffice
Grid Scheduler
Grid Node
Grid Node
Grid Node

contributions,
•Rule based system •User Profile •Local User Profile
•Automated formatting •Dynamic User Profile actions on
•Local Dynamic User Profile
•Inferential engine •User behavior •Local User behavior content,
processing •Use data •Local Use data social actions,
•Adaptation •Content •Content preferences,
•enrichement •DC+IDs •DC+IDs queries,
•AXInfo: ver, prod., •AXInfo: ver, prod, rights,
•Multilingual index and use data,..
rights,.. ....
search •Descriptors •Descriptors
•Text Analysers •Groups: users, content.. •Groups
•Indexer •Ontology/Taxonomy Domain •Taxonomy classification
•Fuzzy search

•Suggestions •Suggestions on the basis of:
•Similarity distances • Static and dynamic user
•Clustering profile, decriptors, domain Content Organiser
•Local Suggestions on the
basis of user profiles, local
AXCP BackOffice Front End Portal content, local collected data

Content Organizer and Players Users
56

 IPR and access management
 User behaviour monitoring and analysis
 Social network information workflow vs user stimuli analysis
 Relationships and interoperability with other SNs
 Content Workflow Management and process
 Natural language processing of user comments
 Deployed Hardware Architecture
 From ECLAP v0 to now….

 Other Social Networks:
 http://www.apretoscana.org
 http://iuf.csavri.org
 http://mobmed.axmedis.org + 1
57

 http://www.eclap.eu
 This slide link: http://bpnet.eclap.eu/drupal/?q=en-
US/home&axoid=urn:axmedis:00000:obj:6f371ec5-b5d5-46e5-bd5e-a10a3e4228b4
 ECLAP workflow and IPR wizard tools can be taken from the ECLAP user
manual:
 http://bpnet.eclap.eu/drupal/?q=en-
US/home&axoid=urn:axmedis:00000:obj:b828710e-b77c-4074-993c-
3efddfbfaad7&section=pagesupport
 Online manual/help:
 http://bpnet.eclap.eu/help/eclap_bpnet_user_manual_v1-0.htm
 Video of the Content Organizer:
US/home&axoid=urn:axmedis:00000:obj:ea5eb861-18b6-4262-8d4b-
77b3d3c084f0
 ECLAP infrastructure document
US/home&axoid=urn:axmedis:00000:obj:a345a84f-6fdf-4f84-a412-
88094ce363e2
58

 A. Bellandi, P. Bellini, A. Cappuccio, P. Nesi, G. Pantaleo, N. Rauch, "ASSISTED KNOWLEDGE
BASE GENERATION, MANAGEMENT AND COMPETENCE RETRIEVAL", International Journal of
Software Engineering and Knowledge Engineering, World Scientific Publishing Company, in press,
2012, 2013
 P. Bellini, I. Bruno, D. Cenni, P. Nesi, "Micro grids for scalable media computing and intelligence on
distributed scenarious", IEEE Multimedia, Feb 2012, Vol.19, N.2, pp.69.79, IEEE Press.
 P. Bellini, I. Bruno, D. Cenni, A. Fuzier, P. Nesi, M. Paolucci, "Mobile Medicine: Semantic Computing
Management for Health Care Applications on Desktop and Mobile Devices", on Multimedia Tools and
Applications, Springer, May 2012, Volume 58, Issue 1, pp 41-79
 P. Bellini, I. Bruno, P. Nesi, "EXPLOITING INTELLIGENT CONTENT VIA AXMEDIS/MPEG-21 FOR
MODELLING AND DISTRIBUTING NEWS", International Journal of Software Engineering and
Knowledge Engineering, World Scientific Publishing Company, 2011, Vol.21, n.1, pp.3-32
 D. Cenni, P. Nesi,. M. Paolucci, "Functionalities and Flow Analyses of Knowledge Oriented Web
Portals", Proceeding of the 18th International Conference on Distributed Multimedia Systems, DMS
2012, Eden Roc Renaissance Miami Beach, USA, SEKE software engineering and knowledge
engineering institute, 9-11 August, 2012.
 P. Belllini, D. Cenni, P. Nesi, "On the Effectiveness and Optimization of Information Retrieval for
Cross Media Content", Proceeding of the KDIR 2012 is part of IC3K 2012, International Joint
Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, 4-7
October 2012, Barcelona, Spain.


 Prof. Paolo Nesi, Ph.D.
DISIT-DSI, Department of Systems and Informatics
Distributed Systems and Internet Technology Lab
University of Florence
Via S. Marta 3, 50139 Firenze, Italy
Email: paolo.nesi@unifi.it
Tel: 0039-055-4796532
Cell: 0039-335-5668674
Skype: paolonesi

60

Anatomy of a Cross Media Best Practice Network for Media Aggregation and Fruition

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (14)

Similar to Anatomy of a Cross Media Best Practice Network for Media Aggregation and Fruition

Similar to Anatomy of a Cross Media Best Practice Network for Media Aggregation and Fruition (20)

Recently uploaded

Recently uploaded (20)

Anatomy of a Cross Media Best Practice Network for Media Aggregation and Fruition