SlideShare a Scribd company logo
Combining Multimedia and Semantics Oscar Corcho (ocorcho@fi.upm.es) Universidad Politécnica de Madrid http://www.oeg-upm.net/ LACNEM 2010, Cali, ColombiaSeptember 9th 2010 Credits: Adrián Siles, Mariano Rico, Víctor Méndez, Hector Andrés García-Silva, María del Carmen Suárez-Figueroa, Ghislain Atemezing, Raphaël Troncy WorkdistributedunderthelicenseCreativeCommonsAttribution-Noncommercial-Share Alike3.0 http://www.slideshare.net/ocorcho
2 Asunción Gómez Pérez OntologyEngineering Group. Whomwe are Director: A. Gómez-Pérez Research Group (37 people) 2 Full Professor 4 AssociateProfessors 1 AssistantProfessor 3 Postdocs 17 PhD Students 8 MScStudents 2 Software Engineers  Management (4 people) 2 Project Managers 1 SystemAdministrator 1 Secretary  50+ PastCollaborators  10+ visitors
Research Areas 2004 2008 1995 1997 2000
Beforewestart… Howmany of youhaveeverheardabouttheword “Ontology”? And howmany of you do actuallyknowwhatitmeans? 4
Comingtotermswithontologies and semantics An ontology is an engineering artifact, which provides:  A vocabulary of terms A set of explicit assumptions regarding the intended meaning of the vocabulary.  Almost always including concepts and their classification Almost always including properties between concepts Shared understanding of a domain of interest  Agreement on the meaning of terms Formal and machine manipulable model of a domain of interest Besides... The meaning (semantics) of such terms is formally specified New terms can be formed by combining existing ones Can also specify relationships between terms in multiple ontologies 5
Example: Anontologyaboutsatellites 6
Outline Introduction What I willbetalkingabout and what I willnot… Therewereseveraloptionsthat I exploredbeforeselectingtheonethatyouwillbehearing in thistalk… 7
Option 1: The Semantic Gap The lack of coincidencebetweentheinformationthatone can extractfromthesensory data and theinterpretationthatthesame data has for a user in a givensituation 8 A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain: Content-based image retrieval at the end of the early years, IEEE PAMI, 1349–1380, 2000. However, I alreadyassumedthatEbroulwouldbetalking a lotaboutit in hisopeningkeynote (as he did).  Besides, I havenotworked at allonthelow-levelpart, so itmaybedifficultfor me toprovideyouwith a goodinsightonthe (many) open problems in thisarea
Option 2: MPEG-7 and the Semantic Web ISO standard since December 2001 Main components: Descriptors (Ds) and Description Schemes (DSs) DDL (XML Schema + extensions) Concern all types of media A good number of ontologies developed around it 9 Part 5 – MDSMultimedia Description Schemes
Option 2: MPEG-7 and the Semantic Web However, thetalkmay: 	Be a bit boring and tootechnical Maylackthemix of state of the art and visionthataninvitedtalkshouldnormallyhave 	And MPEG-7 isnotusedtoomuch …So I willcoveronlysomeaspects of thislater, when I talkabout multimedia ontologies. 10
Option 3: Canonical Processes of Media Production (and semantics, obviously) For example…. http://www.cewe-photobook.com Application for authoring digital photo books Automatic selection, sorting and ordering of photos Context analysis methods: timestamp, annotation, etc. Content analysis methods: color histograms, edge detection, etc. Customized layout and background Print by the European leader photo finisher company 11 Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes My winter ski holidays with my friends Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
Semantics can be important in the process Credits: Raphaël Troncy, LyndaHardman
Option 3: Canonical Processes of Media Production However, some of youprobablyattended Raphaël Troncy’stalklastyear (available in slideshare) 19
In summary… I decidedtotalkaboutsomethingthat I havebeenworking in forthelastcouple of years, and which combines Semantics (of course, thisisthekeyexpertise of ourgroup) Mainlyannotation, Linked Data and a bit of Multimedia OntologyEngineering Social networks, collaboration, sharing and collectiveintelligence Exploiting home networks and online multimedia sites And, obviously, multimedia And hence I stillleaveoutmanyinterestingtopics (e.g., semanticsin user interfaces) 20
Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Credits: Mariano Rico and Adrián Siles (UPM), Víctor Méndez and José Manuel Gómez-Pérez (iSOCO), José Manuel Palacios and Mónica Pérez (TID) Sem4Tags Tagdisambiguation in Flickr M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 21
Internet Motivation Multimedia resources in Web2.0 are stored in centralised servers. You lose some of yourrights as anauthorwhenyouuploadtheseresourcestothese servers. Privacyproblems. Poorannotations and metadata. Theseresourcescannotbesharedwithotherresources in your home. 22 UpGrid
Multimedia Content SharingwithUpGrid 23 Annotation: “Ángel onthebeach” Reasoning: ,[object Object]
 “Pedro is my brother”
 “Juan is my brother”
-------------------------------
Ángel is my nephewJuan P2P Semantic-basedquery: “multimedia contentrelatedto my nephew” Annotation: “Ángel playing soccer” Pedro Additionalsemanticinformation: ,[object Object]
 “Pedro is my brother”Additionalsemanticinformation: “Juan is my brother”
Architecture
Architecture (anotherviewonit)
Snapshotsfromtheapplication Checkhttp://www.youtube.com/results?search_query=UPnPGrid
Summary Aneffectivemeansforsharing multimedia contentsacrosshomes, avoiding Web2.0 siteswhereyourrightsmaybecompromised However, itisstill a prototype, and no serioususabilitytesting has been done Muchworkstillneeded in ordertogointo a real system And endusersfinditdifficulttoprovideannotations Do you imagine yourparents and grandparentsannotatingphotos and videos likethat? Let’sseehowthiscouldbeamelioratedwiththenextpart of ourpresentation. 27
Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Sem4Tags Tagdisambiguation in Flickr Credits: Héctor Andrés García Silva M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 28 Egresado de laUniversidad del Valle
Introduction Social Tagging Systems Web 2.0 applications  Applications for storing, sharing, and discovering information resources. Users assign tagsto identify information resources Tags are used to search/discover resources 29
Introduction Folksonomy Emerging classification scheme from social tagging systems  Folk: People, Taxonomy: Classification Represented by: Users, Tags, Resources Taxonomy Folksonomy ,[object Object]
Controlled Vocabulary
Hierarchical structure
Exclusive/Restrictive
Expensive to maintain
Bottom-up (user created)
No fixed vocabulary
No Hierarchical structure
No Exclusive/Flexible
Low cost30
Introduction Why is tagging so popular? Reduce cognitive burdens  it’s easy to use Users don´t need any special skill or experience The benefits of tagging are immediate Future retrieval Contribution and sharing Attract Attention Self Presentation Opinion Expression 31
Introduction However Tags can be ambiguous  Polysemy: partyas a celebration as opposed to partyas a political organization Synonym: party and celebration  Morphological variations:      party, parties, partying, partyign Plurals Acronyms Conjugated verbs Misspelling Compound words Political party, PoliticalParty, Political_party,     Political-Party, etc. Detail/granularity level A general tag as partyin contrast to a specific tag as banquet. 32
Motivation The problem: Morphological variations, synonyms, granularity, and polysemy hamper information retrieval processes based on folksonomies.  Systems ignore resources tagged with morphological variationsor synonyms of that tag, as well as the resources tagged with more generic or more specific tags 710.659 results 8.661.581 Results 33
    When searching with polysemous tags, all the resources tagged with that tag are retrieved without taking into account the tag sense the user was looking for. 	(e.g., Query flickr with bank results in photos about financial institutions, river edges, fog banks, and sand banks, etc. ) 34 Motivation
Motivation What if we associate tags with semantic entities? http://morpheus.cs.umbc.edu/aks1/ontosem.owl  #non-work-activity We can avoid the aforementioned pitfalls #organization #special-occasion #political-entity #party #Celebration #political-party #Coalition #federation #Birthday #Anniversary uk, tories, party, conservative, speech  party, balloons, colors, bar, crowd 35
State of the Art: Semantic Grounding of Cross-Lingual Folksonomies Garcia HA, Corcho O, Alani H, Gómez-Pérez A. Review of the state of the art: Discovering and Associating Semantics to Folksonomies. Knowledge Engineering Review (in press) None of the analyzed approaches deals with multilingual tags 36
Semantic Grounding of Cross-Lingual Folksonomies MSR: a Multilingual Sense Repository based on Wikipedia and enriched with semantic information taken from DBpedia. Terms and  frequency Banco Bank http://dbpedia.org/resource/Bank Terms and  frequency Banco Cardumen Swarm http://dbpedia.org/resource/Swarm Banco de  Arena http://dbpedia.org/resource/ SandBank Terms and  frequency Sandbank 37
Semantic Grounding of Cross-Lingual Folksonomies Sem4Tags: A process for Associating Semantics to Tags. 38 Dinero, Calle, Santander, Money, Madrid, Atm,  cajero Europe Euro  Finance Central bank awesomePic Nikon .. Bank Banco http://dbpedia.org/resource/Bank
Semantic Grounding of Cross-Lingual Folksonomies Disambiguation activity The candidate senses and the tag context are represented as vectors.  The vector components are the set of most frequent terms in each Wikipedia page representing a sense. For each sense the values of the vector are calculated using TF-IDF. For the tag context the values in each position are 1 or 0 if the corresponding term appears in the tag context.  The tag context vector is compared against each sense vector using the cosine of the angle as similarity measure.  The most similar sense to the tag context is selected as the one representing the meaning of the analyzed tag 39 39
Semantic Grounding of Cross-Lingual Folksonomies Disambiguation activity We use the information of the wikipedia default sense for a term.  Sim(TagContext, Sensei)= λ*Cosine + β*defaultSense We experimentally defined β = 0,2 and λ = 0.8 We attempt to use DBpedia semantic information in the disambiguation activity: Sim(TagContext, Sensei)= λ*Cosine + β*defaultSense + δ*SemanticInfo Studies have shown that tags in flickr refers mainly to: Locations, Time, Given Names, Potography related subjects among others.  We use DBpedia and YAGO relations to classify the senses according to this categories. However, we found that not all the senses related to a term have the same amount of relations. (e.g. Madrid is not a city) 40
Let’s try it http://robinson.dia.fi.upm.es:8080/SemanticTagsWebApp/index.jsp Whatdoes “bernabeu” mean ifitscontextis…? estadio, madrid, fútbol 41
Experiment Baseline: Directly associate tags with DBpedia resources Look for spaces and replace them with  ' _‘. For tags in English: Create a URI of the form http://en.wikipedia.org/wiki/tag Query DBpedia using the http://xmlns.com/foaf/0.1/page relation For tags in Spanish: Create a URI of the form http://es.wikipedia.org/wiki/tag Query DBpedia using the http://dbpedia.org/property/wikipage-es relation 42
Experiment Approaches: Baseline: Selection of the sense without a disambiguation activity. Sem4Tags: For each sense we use the whole Wikipedia article as source for frequentterms. Sem4TagsAC: Same as Sem4Tags including the selection of the Active Context. Sem4TagsAbs: For each sense we use the Wikipedia article abstract (extracted from DBpedia) as source for frequent terms. Sem4TagsAbsAC: Same as Sem4TagsAbs including the selection of the Active Context. 43
Experiment Initial Data Set Wide range of Users, photos, and tags. 764 photos uploaded by 719 users to Flickr that have been tagged with tags describing tourist places in Spain 12.4 (+/- 7.85) tags per photo 9484 tagging activities (TAS) : <user,photo,tag> 4135 distinct tags where used Processed Data Set From each photo we processed on average 2 tags  2260 taggingactivities (TAS) 44
Experiment Evaluation Campaign 41 Evaluators Evaluate semantic associations produce by each approach: <user; tag; photo; DBpedia resource; language> Three different evaluators evaluated each semantic association. Questions: Able to identify the tag meaning (known or Unknown) Tag language (English, Spanish, Both, other) The tag correspond to a Named entity According to the identified tag language they evaluate the semantic association in terms of Highly related, Related, Not Related. 45
Experiment Results Evaluators identified the semantics of the 87% of TAS (known) 62.6 % of TAS were considered in English 87.7% of TAS were considered in Spanish Agreement among evaluators (Fleiss’ kappa statistics): k=0.76 for highly related K=0.71 for the related case/highly related case 46
Experiment Precision and RecallforHighlyRelevantresults 47 English Spanish
Experiment Conclusions Baseline obtained high precision, however it was able to find semantic resources for just a fraction of the analyzed data set: Baseline: 27.7% in English and 19.4% in Spanish. Sem4Tags: 79.1 % in English and 81.4% in Spanish All approaches obtained better precision with named entities than with unnamed entities.  Sem4Tags and Sem4TagsAC are the approaches that obtained the best results in terms of Precision and Recall.  Sometimes Sem4TagsAC obtains better P@1 values but the improvements are supported by no or low statistical evidence.  Sem4TagsAbs and Sem4TagsAbs are clearly the worst approaches.  48
Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Sem4Tags Tagdisambiguation in Flickr M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 49
50 Ontología M3
There are already multimedia ontologies MDS Upper Layer represented in RDFS 2001: Hunter Later on: link to the ABC upper ontology MDS fully represented in OWL-DL 2004: Tsinaraki et al., DS-MIRF model MPEG-7 fully represented in OWL-DL 2005: Garcia and Celma, Rhizomik model Fully automatic translation of the whole standard MDS and Visual parts represented in OWL-DL 2007: Arndt et al., COMM model  Re-engineering MPEG-7 using DOLCE design patterns However, their requirements are not always clear nor have they been developed with clear methodological guidelines

More Related Content

Similar to Combining Multimedia and Semantics (LACNEM2010)

Web 20 E Oltre 1202297800291589 3
Web 20 E Oltre 1202297800291589 3Web 20 E Oltre 1202297800291589 3
Web 20 E Oltre 1202297800291589 3
Universita' di Bari
 
Anatomy of Social Networks, a guide for social media strategists
Anatomy of Social Networks, a guide for social media strategistsAnatomy of Social Networks, a guide for social media strategists
Anatomy of Social Networks, a guide for social media strategists
Paolo Nesi
 
A Semantic Multimedia Web: Create, Annotate, Present and Share your Media
A Semantic Multimedia Web: Create, Annotate, Present and Share your MediaA Semantic Multimedia Web: Create, Annotate, Present and Share your Media
A Semantic Multimedia Web: Create, Annotate, Present and Share your Media
Raphael Troncy
 
myExperiment @ Nettab
myExperiment @ NettabmyExperiment @ Nettab
myExperiment @ Nettab
Duncan Hull
 
Multimedia Semantics - SSMS 2010
Multimedia Semantics - SSMS 2010Multimedia Semantics - SSMS 2010
Multimedia Semantics - SSMS 2010
Raphael Troncy
 
A Semantic Multimedia Web (Part 3)
A Semantic Multimedia Web (Part 3)A Semantic Multimedia Web (Part 3)
A Semantic Multimedia Web (Part 3)
Raphael Troncy
 
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
Amit Sheth
 
Participatory Media Literacy Diverse2008
Participatory Media Literacy Diverse2008Participatory Media Literacy Diverse2008
Participatory Media Literacy Diverse2008
urauch
 
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
Artificial Intelligence Institute at UofSC
 
Semantic Web in Action
Semantic Web in ActionSemantic Web in Action
Semantic Web in Action
Sebastian Ryszard Kruk
 
Visual Information Retrieval: Advances, Challenges and Opportunities
Visual Information Retrieval: Advances, Challenges and OpportunitiesVisual Information Retrieval: Advances, Challenges and Opportunities
Visual Information Retrieval: Advances, Challenges and Opportunities
Oge Marques
 
Learning as a Social Process
Learning as a Social ProcessLearning as a Social Process
Learning as a Social Process
Robert Cormia
 
SDA2013 Pundit: Creating, Exploring and Consuming Annotations
SDA2013 Pundit: Creating, Exploring and Consuming AnnotationsSDA2013 Pundit: Creating, Exploring and Consuming Annotations
SDA2013 Pundit: Creating, Exploring and Consuming Annotations
Marco Grassi
 
WP2 1st Review
WP2 1st ReviewWP2 1st Review
WP2 1st Review
INSEMTIVES project
 
Breaking Down Walls in Enterprise with Social Semantics
Breaking Down Walls in Enterprise with Social SemanticsBreaking Down Walls in Enterprise with Social Semantics
Breaking Down Walls in Enterprise with Social Semantics
John Breslin
 
Intelligentcontent2009
Intelligentcontent2009Intelligentcontent2009
Intelligentcontent2009
Salim Ismail
 
Gic2011 aula0-ingles
Gic2011 aula0-inglesGic2011 aula0-ingles
Gic2011 aula0-ingles
Marielba-Mayeya Zacarias
 
Gettingstartedwithdigitalcollectionsweb[1]
Gettingstartedwithdigitalcollectionsweb[1]Gettingstartedwithdigitalcollectionsweb[1]
Gettingstartedwithdigitalcollectionsweb[1]
guest410707c
 
Complex Networks: Science, Programming, and Databases
Complex Networks: Science, Programming, and DatabasesComplex Networks: Science, Programming, and Databases
Complex Networks: Science, Programming, and Databases
S.M. Mahdi Seyednezhad, Ph.D.
 
CFP-Word
CFP-WordCFP-Word
CFP-Word
butest
 

Similar to Combining Multimedia and Semantics (LACNEM2010) (20)

Web 20 E Oltre 1202297800291589 3
Web 20 E Oltre 1202297800291589 3Web 20 E Oltre 1202297800291589 3
Web 20 E Oltre 1202297800291589 3
 
Anatomy of Social Networks, a guide for social media strategists
Anatomy of Social Networks, a guide for social media strategistsAnatomy of Social Networks, a guide for social media strategists
Anatomy of Social Networks, a guide for social media strategists
 
A Semantic Multimedia Web: Create, Annotate, Present and Share your Media
A Semantic Multimedia Web: Create, Annotate, Present and Share your MediaA Semantic Multimedia Web: Create, Annotate, Present and Share your Media
A Semantic Multimedia Web: Create, Annotate, Present and Share your Media
 
myExperiment @ Nettab
myExperiment @ NettabmyExperiment @ Nettab
myExperiment @ Nettab
 
Multimedia Semantics - SSMS 2010
Multimedia Semantics - SSMS 2010Multimedia Semantics - SSMS 2010
Multimedia Semantics - SSMS 2010
 
A Semantic Multimedia Web (Part 3)
A Semantic Multimedia Web (Part 3)A Semantic Multimedia Web (Part 3)
A Semantic Multimedia Web (Part 3)
 
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
Semantic Web & Information Brokering: Opportunities, Commercialization and Ch...
 
Participatory Media Literacy Diverse2008
Participatory Media Literacy Diverse2008Participatory Media Literacy Diverse2008
Participatory Media Literacy Diverse2008
 
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
Relationships at the Heart of Semantic Web: Modeling, Discovering, Validating...
 
Semantic Web in Action
Semantic Web in ActionSemantic Web in Action
Semantic Web in Action
 
Visual Information Retrieval: Advances, Challenges and Opportunities
Visual Information Retrieval: Advances, Challenges and OpportunitiesVisual Information Retrieval: Advances, Challenges and Opportunities
Visual Information Retrieval: Advances, Challenges and Opportunities
 
Learning as a Social Process
Learning as a Social ProcessLearning as a Social Process
Learning as a Social Process
 
SDA2013 Pundit: Creating, Exploring and Consuming Annotations
SDA2013 Pundit: Creating, Exploring and Consuming AnnotationsSDA2013 Pundit: Creating, Exploring and Consuming Annotations
SDA2013 Pundit: Creating, Exploring and Consuming Annotations
 
WP2 1st Review
WP2 1st ReviewWP2 1st Review
WP2 1st Review
 
Breaking Down Walls in Enterprise with Social Semantics
Breaking Down Walls in Enterprise with Social SemanticsBreaking Down Walls in Enterprise with Social Semantics
Breaking Down Walls in Enterprise with Social Semantics
 
Intelligentcontent2009
Intelligentcontent2009Intelligentcontent2009
Intelligentcontent2009
 
Gic2011 aula0-ingles
Gic2011 aula0-inglesGic2011 aula0-ingles
Gic2011 aula0-ingles
 
Gettingstartedwithdigitalcollectionsweb[1]
Gettingstartedwithdigitalcollectionsweb[1]Gettingstartedwithdigitalcollectionsweb[1]
Gettingstartedwithdigitalcollectionsweb[1]
 
Complex Networks: Science, Programming, and Databases
Complex Networks: Science, Programming, and DatabasesComplex Networks: Science, Programming, and Databases
Complex Networks: Science, Programming, and Databases
 
CFP-Word
CFP-WordCFP-Word
CFP-Word
 

More from Oscar Corcho

Organisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de MadridOrganisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de Madrid
Oscar Corcho
 
Introducción a los Datos Abiertos - Open Data Day 2020
Introducción a los Datos Abiertos - Open Data Day 2020Introducción a los Datos Abiertos - Open Data Day 2020
Introducción a los Datos Abiertos - Open Data Day 2020
Oscar Corcho
 
Open Data (and Software, and other Research Artefacts) - A proper management
Open Data (and Software, and other Research Artefacts) -A proper managementOpen Data (and Software, and other Research Artefacts) -A proper management
Open Data (and Software, and other Research Artefacts) - A proper management
Oscar Corcho
 
Adiós a los ficheros, hola a los grafos de conocimientos estadísticos
Adiós a los ficheros, hola a los grafos de conocimientos estadísticosAdiós a los ficheros, hola a los grafos de conocimientos estadísticos
Adiós a los ficheros, hola a los grafos de conocimientos estadísticos
Oscar Corcho
 
Ontology Engineering at Scale for Open City Data Sharing
Ontology Engineering at Scale for Open City Data SharingOntology Engineering at Scale for Open City Data Sharing
Ontology Engineering at Scale for Open City Data Sharing
Oscar Corcho
 
Situación de las iniciativas de Open Data internacionales (y algunas recomen...
Situación de las iniciativas de Open Data internacionales (y algunas recomen...Situación de las iniciativas de Open Data internacionales (y algunas recomen...
Situación de las iniciativas de Open Data internacionales (y algunas recomen...
Oscar Corcho
 
STARS4ALL - Contaminación Lumínica
STARS4ALL - Contaminación LumínicaSTARS4ALL - Contaminación Lumínica
STARS4ALL - Contaminación Lumínica
Oscar Corcho
 
Towards Reproducible Science: a few building blocks from my personal experience
Towards Reproducible Science: a few building blocks from my personal experienceTowards Reproducible Science: a few building blocks from my personal experience
Towards Reproducible Science: a few building blocks from my personal experience
Oscar Corcho
 
Publishing Linked Statistical Data: Aragón, a case study
Publishing Linked Statistical Data: Aragón, a case studyPublishing Linked Statistical Data: Aragón, a case study
Publishing Linked Statistical Data: Aragón, a case study
Oscar Corcho
 
An initial analysis of topic-based similarity among scientific documents base...
An initial analysis of topic-based similarity among scientific documents base...An initial analysis of topic-based similarity among scientific documents base...
An initial analysis of topic-based similarity among scientific documents base...
Oscar Corcho
 
Generación de datos estadísticos enlazados del Instituto Aragonés de Estadística
Generación de datos estadísticos enlazados del Instituto Aragonés de EstadísticaGeneración de datos estadísticos enlazados del Instituto Aragonés de Estadística
Generación de datos estadísticos enlazados del Instituto Aragonés de Estadística
Oscar Corcho
 
Presentación de la red de excelencia de Open Data y Smart Cities
Presentación de la red de excelencia de Open Data y Smart CitiesPresentación de la red de excelencia de Open Data y Smart Cities
Presentación de la red de excelencia de Open Data y Smart Cities
Oscar Corcho
 
Why do they call it Linked Data when they want to say...?
Why do they call it Linked Data when they want to say...?Why do they call it Linked Data when they want to say...?
Why do they call it Linked Data when they want to say...?
Oscar Corcho
 
Linked Statistical Data: does it actually pay off?
Linked Statistical Data: does it actually pay off?Linked Statistical Data: does it actually pay off?
Linked Statistical Data: does it actually pay off?
Oscar Corcho
 
Slow-cooked data and APIs in the world of Big Data: the view from a city per...
Slow-cooked data and APIs in the world of Big Data: the view from a city per...Slow-cooked data and APIs in the world of Big Data: the view from a city per...
Slow-cooked data and APIs in the world of Big Data: the view from a city per...
Oscar Corcho
 
Research Objects for improved sharing and reproducibility
Research Objects for improved sharing and reproducibilityResearch Objects for improved sharing and reproducibility
Research Objects for improved sharing and reproducibility
Oscar Corcho
 
(Big) Data (Science) Skills
(Big) Data (Science) Skills(Big) Data (Science) Skills
(Big) Data (Science) Skills
Oscar Corcho
 
Big Data - El Futuro a través de los Datos
Big Data - El Futuro a través de los DatosBig Data - El Futuro a través de los Datos
Big Data - El Futuro a través de los Datos
Oscar Corcho
 
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
Oscar Corcho
 
Aspectos técnicos de la ontología PPROC
Aspectos técnicos de la ontología PPROCAspectos técnicos de la ontología PPROC
Aspectos técnicos de la ontología PPROC
Oscar Corcho
 

More from Oscar Corcho (20)

Organisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de MadridOrganisational Interoperability in Practice at Universidad Politécnica de Madrid
Organisational Interoperability in Practice at Universidad Politécnica de Madrid
 
Introducción a los Datos Abiertos - Open Data Day 2020
Introducción a los Datos Abiertos - Open Data Day 2020Introducción a los Datos Abiertos - Open Data Day 2020
Introducción a los Datos Abiertos - Open Data Day 2020
 
Open Data (and Software, and other Research Artefacts) - A proper management
Open Data (and Software, and other Research Artefacts) -A proper managementOpen Data (and Software, and other Research Artefacts) -A proper management
Open Data (and Software, and other Research Artefacts) - A proper management
 
Adiós a los ficheros, hola a los grafos de conocimientos estadísticos
Adiós a los ficheros, hola a los grafos de conocimientos estadísticosAdiós a los ficheros, hola a los grafos de conocimientos estadísticos
Adiós a los ficheros, hola a los grafos de conocimientos estadísticos
 
Ontology Engineering at Scale for Open City Data Sharing
Ontology Engineering at Scale for Open City Data SharingOntology Engineering at Scale for Open City Data Sharing
Ontology Engineering at Scale for Open City Data Sharing
 
Situación de las iniciativas de Open Data internacionales (y algunas recomen...
Situación de las iniciativas de Open Data internacionales (y algunas recomen...Situación de las iniciativas de Open Data internacionales (y algunas recomen...
Situación de las iniciativas de Open Data internacionales (y algunas recomen...
 
STARS4ALL - Contaminación Lumínica
STARS4ALL - Contaminación LumínicaSTARS4ALL - Contaminación Lumínica
STARS4ALL - Contaminación Lumínica
 
Towards Reproducible Science: a few building blocks from my personal experience
Towards Reproducible Science: a few building blocks from my personal experienceTowards Reproducible Science: a few building blocks from my personal experience
Towards Reproducible Science: a few building blocks from my personal experience
 
Publishing Linked Statistical Data: Aragón, a case study
Publishing Linked Statistical Data: Aragón, a case studyPublishing Linked Statistical Data: Aragón, a case study
Publishing Linked Statistical Data: Aragón, a case study
 
An initial analysis of topic-based similarity among scientific documents base...
An initial analysis of topic-based similarity among scientific documents base...An initial analysis of topic-based similarity among scientific documents base...
An initial analysis of topic-based similarity among scientific documents base...
 
Generación de datos estadísticos enlazados del Instituto Aragonés de Estadística
Generación de datos estadísticos enlazados del Instituto Aragonés de EstadísticaGeneración de datos estadísticos enlazados del Instituto Aragonés de Estadística
Generación de datos estadísticos enlazados del Instituto Aragonés de Estadística
 
Presentación de la red de excelencia de Open Data y Smart Cities
Presentación de la red de excelencia de Open Data y Smart CitiesPresentación de la red de excelencia de Open Data y Smart Cities
Presentación de la red de excelencia de Open Data y Smart Cities
 
Why do they call it Linked Data when they want to say...?
Why do they call it Linked Data when they want to say...?Why do they call it Linked Data when they want to say...?
Why do they call it Linked Data when they want to say...?
 
Linked Statistical Data: does it actually pay off?
Linked Statistical Data: does it actually pay off?Linked Statistical Data: does it actually pay off?
Linked Statistical Data: does it actually pay off?
 
Slow-cooked data and APIs in the world of Big Data: the view from a city per...
Slow-cooked data and APIs in the world of Big Data: the view from a city per...Slow-cooked data and APIs in the world of Big Data: the view from a city per...
Slow-cooked data and APIs in the world of Big Data: the view from a city per...
 
Research Objects for improved sharing and reproducibility
Research Objects for improved sharing and reproducibilityResearch Objects for improved sharing and reproducibility
Research Objects for improved sharing and reproducibility
 
(Big) Data (Science) Skills
(Big) Data (Science) Skills(Big) Data (Science) Skills
(Big) Data (Science) Skills
 
Big Data - El Futuro a través de los Datos
Big Data - El Futuro a través de los DatosBig Data - El Futuro a través de los Datos
Big Data - El Futuro a través de los Datos
 
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
EKAW2014 Keynote: Ontology Engineering for and by the Masses: are we already ...
 
Aspectos técnicos de la ontología PPROC
Aspectos técnicos de la ontología PPROCAspectos técnicos de la ontología PPROC
Aspectos técnicos de la ontología PPROC
 

Recently uploaded

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
IndexBug
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
名前 です男
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
Uni Systems S.M.S.A.
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
Matthew Sinclair
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
Zilliz
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Malak Abu Hammad
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
Claudio Di Ciccio
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
DianaGray10
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
Octavian Nadolu
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
Mariano Tinti
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
Zilliz
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
SitimaJohn
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
Claudio Di Ciccio
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems S.M.S.A.
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
Techgropse Pvt.Ltd.
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
Daiki Mogmet Ito
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
Ivanti
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
kumardaparthi1024
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
panagenda
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Speck&Tech
 

Recently uploaded (20)

AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceAI 101: An Introduction to the Basics and Impact of Artificial Intelligence
AI 101: An Introduction to the Basics and Impact of Artificial Intelligence
 
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
みなさんこんにちはこれ何文字まで入るの?40文字以下不可とか本当に意味わからないけどこれ限界文字数書いてないからマジでやばい文字数いけるんじゃないの?えこ...
 
Microsoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdfMicrosoft - Power Platform_G.Aspiotis.pdf
Microsoft - Power Platform_G.Aspiotis.pdf
 
20240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 202420240607 QFM018 Elixir Reading List May 2024
20240607 QFM018 Elixir Reading List May 2024
 
Generating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and MilvusGenerating privacy-protected synthetic data using Secludy and Milvus
Generating privacy-protected synthetic data using Secludy and Milvus
 
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfUnlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdf
 
CAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on BlockchainCAKE: Sharing Slices of Confidential Data on Blockchain
CAKE: Sharing Slices of Confidential Data on Blockchain
 
UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6UiPath Test Automation using UiPath Test Suite series, part 6
UiPath Test Automation using UiPath Test Suite series, part 6
 
Artificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopmentArtificial Intelligence for XMLDevelopment
Artificial Intelligence for XMLDevelopment
 
Mariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceXMariano G Tinti - Decoding SpaceX
Mariano G Tinti - Decoding SpaceX
 
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalizationFull-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
 
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptxOcean lotus Threat actors project by John Sitima 2024 (1).pptx
Ocean lotus Threat actors project by John Sitima 2024 (1).pptx
 
“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”“I’m still / I’m still / Chaining from the Block”
“I’m still / I’m still / Chaining from the Block”
 
Uni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdfUni Systems Copilot event_05062024_C.Vlachos.pdf
Uni Systems Copilot event_05062024_C.Vlachos.pdf
 
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdfAI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
AI-Powered Food Delivery Transforming App Development in Saudi Arabia.pdf
 
How to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For FlutterHow to use Firebase Data Connect For Flutter
How to use Firebase Data Connect For Flutter
 
June Patch Tuesday
June Patch TuesdayJune Patch Tuesday
June Patch Tuesday
 
GenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizationsGenAI Pilot Implementation in the organizations
GenAI Pilot Implementation in the organizations
 
HCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAUHCL Notes and Domino License Cost Reduction in the World of DLAU
HCL Notes and Domino License Cost Reduction in the World of DLAU
 
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?
 

Combining Multimedia and Semantics (LACNEM2010)

  • 1. Combining Multimedia and Semantics Oscar Corcho (ocorcho@fi.upm.es) Universidad Politécnica de Madrid http://www.oeg-upm.net/ LACNEM 2010, Cali, ColombiaSeptember 9th 2010 Credits: Adrián Siles, Mariano Rico, Víctor Méndez, Hector Andrés García-Silva, María del Carmen Suárez-Figueroa, Ghislain Atemezing, Raphaël Troncy WorkdistributedunderthelicenseCreativeCommonsAttribution-Noncommercial-Share Alike3.0 http://www.slideshare.net/ocorcho
  • 2. 2 Asunción Gómez Pérez OntologyEngineering Group. Whomwe are Director: A. Gómez-Pérez Research Group (37 people) 2 Full Professor 4 AssociateProfessors 1 AssistantProfessor 3 Postdocs 17 PhD Students 8 MScStudents 2 Software Engineers Management (4 people) 2 Project Managers 1 SystemAdministrator 1 Secretary 50+ PastCollaborators 10+ visitors
  • 3. Research Areas 2004 2008 1995 1997 2000
  • 4. Beforewestart… Howmany of youhaveeverheardabouttheword “Ontology”? And howmany of you do actuallyknowwhatitmeans? 4
  • 5. Comingtotermswithontologies and semantics An ontology is an engineering artifact, which provides: A vocabulary of terms A set of explicit assumptions regarding the intended meaning of the vocabulary. Almost always including concepts and their classification Almost always including properties between concepts Shared understanding of a domain of interest Agreement on the meaning of terms Formal and machine manipulable model of a domain of interest Besides... The meaning (semantics) of such terms is formally specified New terms can be formed by combining existing ones Can also specify relationships between terms in multiple ontologies 5
  • 7. Outline Introduction What I willbetalkingabout and what I willnot… Therewereseveraloptionsthat I exploredbeforeselectingtheonethatyouwillbehearing in thistalk… 7
  • 8. Option 1: The Semantic Gap The lack of coincidencebetweentheinformationthatone can extractfromthesensory data and theinterpretationthatthesame data has for a user in a givensituation 8 A.W.M. Smeulders, M. Worring, S. Santini, A. Gupta, R. Jain: Content-based image retrieval at the end of the early years, IEEE PAMI, 1349–1380, 2000. However, I alreadyassumedthatEbroulwouldbetalking a lotaboutit in hisopeningkeynote (as he did). Besides, I havenotworked at allonthelow-levelpart, so itmaybedifficultfor me toprovideyouwith a goodinsightonthe (many) open problems in thisarea
  • 9. Option 2: MPEG-7 and the Semantic Web ISO standard since December 2001 Main components: Descriptors (Ds) and Description Schemes (DSs) DDL (XML Schema + extensions) Concern all types of media A good number of ontologies developed around it 9 Part 5 – MDSMultimedia Description Schemes
  • 10. Option 2: MPEG-7 and the Semantic Web However, thetalkmay: Be a bit boring and tootechnical Maylackthemix of state of the art and visionthataninvitedtalkshouldnormallyhave And MPEG-7 isnotusedtoomuch …So I willcoveronlysomeaspects of thislater, when I talkabout multimedia ontologies. 10
  • 11. Option 3: Canonical Processes of Media Production (and semantics, obviously) For example…. http://www.cewe-photobook.com Application for authoring digital photo books Automatic selection, sorting and ordering of photos Context analysis methods: timestamp, annotation, etc. Content analysis methods: color histograms, edge detection, etc. Customized layout and background Print by the European leader photo finisher company 11 Credits: Raphaël Troncy, LyndaHardman
  • 12. CeWe Color PhotoBook Processes My winter ski holidays with my friends Credits: Raphaël Troncy, LyndaHardman
  • 13. CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
  • 14. CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
  • 15. CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
  • 16. CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
  • 17. CeWe Color PhotoBook Processes Credits: Raphaël Troncy, LyndaHardman
  • 18. Semantics can be important in the process Credits: Raphaël Troncy, LyndaHardman
  • 19. Option 3: Canonical Processes of Media Production However, some of youprobablyattended Raphaël Troncy’stalklastyear (available in slideshare) 19
  • 20. In summary… I decidedtotalkaboutsomethingthat I havebeenworking in forthelastcouple of years, and which combines Semantics (of course, thisisthekeyexpertise of ourgroup) Mainlyannotation, Linked Data and a bit of Multimedia OntologyEngineering Social networks, collaboration, sharing and collectiveintelligence Exploiting home networks and online multimedia sites And, obviously, multimedia And hence I stillleaveoutmanyinterestingtopics (e.g., semanticsin user interfaces) 20
  • 21. Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Credits: Mariano Rico and Adrián Siles (UPM), Víctor Méndez and José Manuel Gómez-Pérez (iSOCO), José Manuel Palacios and Mónica Pérez (TID) Sem4Tags Tagdisambiguation in Flickr M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 21
  • 22. Internet Motivation Multimedia resources in Web2.0 are stored in centralised servers. You lose some of yourrights as anauthorwhenyouuploadtheseresourcestothese servers. Privacyproblems. Poorannotations and metadata. Theseresourcescannotbesharedwithotherresources in your home. 22 UpGrid
  • 23.
  • 24. “Pedro is my brother”
  • 25. “Juan is my brother”
  • 27.
  • 28. “Pedro is my brother”Additionalsemanticinformation: “Juan is my brother”
  • 32. Summary Aneffectivemeansforsharing multimedia contentsacrosshomes, avoiding Web2.0 siteswhereyourrightsmaybecompromised However, itisstill a prototype, and no serioususabilitytesting has been done Muchworkstillneeded in ordertogointo a real system And endusersfinditdifficulttoprovideannotations Do you imagine yourparents and grandparentsannotatingphotos and videos likethat? Let’sseehowthiscouldbeamelioratedwiththenextpart of ourpresentation. 27
  • 33. Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Sem4Tags Tagdisambiguation in Flickr Credits: Héctor Andrés García Silva M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 28 Egresado de laUniversidad del Valle
  • 34. Introduction Social Tagging Systems Web 2.0 applications Applications for storing, sharing, and discovering information resources. Users assign tagsto identify information resources Tags are used to search/discover resources 29
  • 35.
  • 45. Introduction Why is tagging so popular? Reduce cognitive burdens it’s easy to use Users don´t need any special skill or experience The benefits of tagging are immediate Future retrieval Contribution and sharing Attract Attention Self Presentation Opinion Expression 31
  • 46. Introduction However Tags can be ambiguous Polysemy: partyas a celebration as opposed to partyas a political organization Synonym: party and celebration  Morphological variations: party, parties, partying, partyign Plurals Acronyms Conjugated verbs Misspelling Compound words Political party, PoliticalParty, Political_party, Political-Party, etc. Detail/granularity level A general tag as partyin contrast to a specific tag as banquet. 32
  • 47. Motivation The problem: Morphological variations, synonyms, granularity, and polysemy hamper information retrieval processes based on folksonomies. Systems ignore resources tagged with morphological variationsor synonyms of that tag, as well as the resources tagged with more generic or more specific tags 710.659 results 8.661.581 Results 33
  • 48. When searching with polysemous tags, all the resources tagged with that tag are retrieved without taking into account the tag sense the user was looking for. (e.g., Query flickr with bank results in photos about financial institutions, river edges, fog banks, and sand banks, etc. ) 34 Motivation
  • 49. Motivation What if we associate tags with semantic entities? http://morpheus.cs.umbc.edu/aks1/ontosem.owl  #non-work-activity We can avoid the aforementioned pitfalls #organization #special-occasion #political-entity #party #Celebration #political-party #Coalition #federation #Birthday #Anniversary uk, tories, party, conservative, speech party, balloons, colors, bar, crowd 35
  • 50. State of the Art: Semantic Grounding of Cross-Lingual Folksonomies Garcia HA, Corcho O, Alani H, Gómez-Pérez A. Review of the state of the art: Discovering and Associating Semantics to Folksonomies. Knowledge Engineering Review (in press) None of the analyzed approaches deals with multilingual tags 36
  • 51. Semantic Grounding of Cross-Lingual Folksonomies MSR: a Multilingual Sense Repository based on Wikipedia and enriched with semantic information taken from DBpedia. Terms and frequency Banco Bank http://dbpedia.org/resource/Bank Terms and frequency Banco Cardumen Swarm http://dbpedia.org/resource/Swarm Banco de Arena http://dbpedia.org/resource/ SandBank Terms and frequency Sandbank 37
  • 52. Semantic Grounding of Cross-Lingual Folksonomies Sem4Tags: A process for Associating Semantics to Tags. 38 Dinero, Calle, Santander, Money, Madrid, Atm, cajero Europe Euro Finance Central bank awesomePic Nikon .. Bank Banco http://dbpedia.org/resource/Bank
  • 53. Semantic Grounding of Cross-Lingual Folksonomies Disambiguation activity The candidate senses and the tag context are represented as vectors. The vector components are the set of most frequent terms in each Wikipedia page representing a sense. For each sense the values of the vector are calculated using TF-IDF. For the tag context the values in each position are 1 or 0 if the corresponding term appears in the tag context. The tag context vector is compared against each sense vector using the cosine of the angle as similarity measure. The most similar sense to the tag context is selected as the one representing the meaning of the analyzed tag 39 39
  • 54. Semantic Grounding of Cross-Lingual Folksonomies Disambiguation activity We use the information of the wikipedia default sense for a term. Sim(TagContext, Sensei)= λ*Cosine + β*defaultSense We experimentally defined β = 0,2 and λ = 0.8 We attempt to use DBpedia semantic information in the disambiguation activity: Sim(TagContext, Sensei)= λ*Cosine + β*defaultSense + δ*SemanticInfo Studies have shown that tags in flickr refers mainly to: Locations, Time, Given Names, Potography related subjects among others. We use DBpedia and YAGO relations to classify the senses according to this categories. However, we found that not all the senses related to a term have the same amount of relations. (e.g. Madrid is not a city) 40
  • 55. Let’s try it http://robinson.dia.fi.upm.es:8080/SemanticTagsWebApp/index.jsp Whatdoes “bernabeu” mean ifitscontextis…? estadio, madrid, fútbol 41
  • 56. Experiment Baseline: Directly associate tags with DBpedia resources Look for spaces and replace them with ' _‘. For tags in English: Create a URI of the form http://en.wikipedia.org/wiki/tag Query DBpedia using the http://xmlns.com/foaf/0.1/page relation For tags in Spanish: Create a URI of the form http://es.wikipedia.org/wiki/tag Query DBpedia using the http://dbpedia.org/property/wikipage-es relation 42
  • 57. Experiment Approaches: Baseline: Selection of the sense without a disambiguation activity. Sem4Tags: For each sense we use the whole Wikipedia article as source for frequentterms. Sem4TagsAC: Same as Sem4Tags including the selection of the Active Context. Sem4TagsAbs: For each sense we use the Wikipedia article abstract (extracted from DBpedia) as source for frequent terms. Sem4TagsAbsAC: Same as Sem4TagsAbs including the selection of the Active Context. 43
  • 58. Experiment Initial Data Set Wide range of Users, photos, and tags. 764 photos uploaded by 719 users to Flickr that have been tagged with tags describing tourist places in Spain 12.4 (+/- 7.85) tags per photo 9484 tagging activities (TAS) : <user,photo,tag> 4135 distinct tags where used Processed Data Set From each photo we processed on average 2 tags 2260 taggingactivities (TAS) 44
  • 59. Experiment Evaluation Campaign 41 Evaluators Evaluate semantic associations produce by each approach: <user; tag; photo; DBpedia resource; language> Three different evaluators evaluated each semantic association. Questions: Able to identify the tag meaning (known or Unknown) Tag language (English, Spanish, Both, other) The tag correspond to a Named entity According to the identified tag language they evaluate the semantic association in terms of Highly related, Related, Not Related. 45
  • 60. Experiment Results Evaluators identified the semantics of the 87% of TAS (known) 62.6 % of TAS were considered in English 87.7% of TAS were considered in Spanish Agreement among evaluators (Fleiss’ kappa statistics): k=0.76 for highly related K=0.71 for the related case/highly related case 46
  • 61. Experiment Precision and RecallforHighlyRelevantresults 47 English Spanish
  • 62. Experiment Conclusions Baseline obtained high precision, however it was able to find semantic resources for just a fraction of the analyzed data set: Baseline: 27.7% in English and 19.4% in Spanish. Sem4Tags: 79.1 % in English and 81.4% in Spanish All approaches obtained better precision with named entities than with unnamed entities. Sem4Tags and Sem4TagsAC are the approaches that obtained the best results in terms of Precision and Recall. Sometimes Sem4TagsAC obtains better P@1 values but the improvements are supported by no or low statistical evidence. Sem4TagsAbs and Sem4TagsAbs are clearly the worst approaches. 48
  • 63. Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Sem4Tags Tagdisambiguation in Flickr M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 49
  • 65. There are already multimedia ontologies MDS Upper Layer represented in RDFS 2001: Hunter Later on: link to the ABC upper ontology MDS fully represented in OWL-DL 2004: Tsinaraki et al., DS-MIRF model MPEG-7 fully represented in OWL-DL 2005: Garcia and Celma, Rhizomik model Fully automatic translation of the whole standard MDS and Visual parts represented in OWL-DL 2007: Arndt et al., COMM model Re-engineering MPEG-7 using DOLCE design patterns However, their requirements are not always clear nor have they been developed with clear methodological guidelines
  • 66. 52 Knowledge Resources Ontological Resources O. Design Patterns 3 4 O. Repositories and Registries 5 6 Flogic RDF(S) OWL Ontological Resource Reuse O. Aligning O. Merging 5 6 2 Ontology Design Pattern Reuse Non Ontological Resource Reuse 4 3 6 Non Ontological Resources 2 Ontological Resource Reengineering 7 Glossaries Dictionaries Lexicons 5 Non Ontological Resource Reengineering 4 6 Classification Schemas Thesauri Taxonomies Alignments 2 RDF(S) 1 Flogic O. Conceptualization O. Implementation O. Formalization O. Specification Scheduling OWL 8 Ontology Restructuring (Pruning, Extension, Specialization, Modularization) 9 O. Localization 1,2,3,4,5,6,7,8, 9 Ontology Support Activities: Knowledge Acquisition (Elicitation); Documentation; Configuration Management; Evaluation (V&V); Assessment NeOnMethodology
  • 67.
  • 68.
  • 69. Content specific requirements referred to the particular knowledge to be represented by the ontology
  • 71. in the form ofCQs
  • 72. in the form of sentences (General Characteristics)
  • 73.
  • 74. 56 Ontology Requirements Specification (IV): ORSD Perspectiva Multidominio M3 Perspectiva Multimedia Perspectiva Multilenguaje
  • 75. 57 Knowledge Resources Ontological Resources O. Design Patterns 3 4 O. Repositories and Registries 5 6 Flogic RDF(S) OWL Ontological Resource Reuse O. Aligning O. Merging 5 6 2 Ontology Design Pattern Reuse Non Ontological Resource Reuse 4 3 6 Non Ontological Resources 2 Ontological Resource Reengineering 7 Glossaries Dictionaries Lexicons 5 Non Ontological Resource Reengineering 4 6 Classification Schemas Thesauri Taxonomies Alignments 2 RDF(S) 1 Flogic O. Conceptualization O. Implementation O. Formalization O. Specification Scheduling OWL 8 Ontology Restructuring (Pruning, Extension, Specialization, Modularization) 9 O. Localization 1,2,3,4,5,6,7,8, 9 Ontology Support Activities: Knowledge Acquisition (Elicitation); Documentation; Configuration Management; Evaluation (V&V); Assessment NeOnMethodology
  • 76. 58 Scheduling using gOntt gOntt Life cycle model selection I need to schedule the development of the M3 ontolgy network Scenarios selection
  • 77. 59 Scheduling using gOntt (II) gOntt
  • 78. 60 Knowledge Resources Ontological Resources O. Design Patterns 3 4 O. Repositories and Registries 5 6 Flogic RDF(S) OWL Ontological Resource Reuse O. Aligning O. Merging 5 6 2 Ontology Design Pattern Reuse Non Ontological Resource Reuse 4 3 6 Non Ontological Resources 2 Ontological Resource Reengineering 7 Glossaries Dictionaries Lexicons 5 Non Ontological Resource Reengineering 4 6 Classification Schemas Thesauri Taxonomies Alignments 2 RDF(S) 1 Flogic O. Conceptualization O. Implementation O. Formalization O. Specification Scheduling OWL 8 Ontology Restructuring (Pruning, Extension, Specialization, Modularization) 9 O. Localization 1,2,3,4,5,6,7,8, 9 Ontology Support Activities: Knowledge Acquisition (Elicitation); Documentation; Configuration Management; Evaluation (V&V); Assessment NeOnMethodology
  • 79. 61 Reusing Ontological Resources: Comparative Analysis (I)
  • 80. 62 Reusing Ontological Resources: Comparative Analysis
  • 81. Outline Introduction What I willbetalkingabout and what I willnot Sem-UPnP-Grid Sharing multimedia contentacrosshomesthroughsemanticannotations Sem4Tags Tagdisambiguation in Flickr M3 Ontology(onlyif time permits) A semanticbackboneforour multimedia-relatedwork Conclusions and outlook 63
  • 82. Conclusions and outlook Weallagreethat… Multimedia UGC has beenone of thebasis of Web2.0 The use of semantics can provide… Betterunderstanding of thedomain and of theircontent Heavyweight: addressingthesemantic gap automatically Ligthweight: allowinguserstoannotate Middleweight: from free tagstoknowledge Betterexploratorynavigation and serendipity Interconnecting multimedia contentwiththeLinked Data cloud However, privacyissues are still a majorbarrierfor a largeruptake, especiallyforsomepopulationsegments Allowing P2P exchangebetween “known” homes, whileexploitingsemantic-basedsearch 64
  • 83. Combining Multimedia and Semantics Oscar Corcho (ocorcho@fi.upm.es) Universidad Politécnica de Madrid http://www.oeg-upm.net/ LACNEM 2010, Cali, ColombiaSeptember 9th 2010 Credits: Adrián Siles, Mariano Rico, Víctor Méndez, Hector Andrés García-Silva, María del Carmen Suárez-Figueroa, Ghislain Atemezing, Raphaël Troncy WorkdistributedunderthelicenseCreativeCommonsAttribution-Noncommercial-Share Alike3.0 http://www.slideshare.net/ocorcho