• Save
Semantic structuring and linking of event-centric data in the social web
Upcoming SlideShare
Loading in...5
×
 

Semantic structuring and linking of event-centric data in the social web

on

  • 1,306 views

Final presentation of the WP4 ALIAS project funded by the EU AAL Programme, see http://www.aal-alias.eu/

Final presentation of the WP4 ALIAS project funded by the EU AAL Programme, see http://www.aal-alias.eu/

Statistics

Views

Total Views
1,306
Views on SlideShare
961
Embed Views
345

Actions

Likes
1
Downloads
0
Comments
0

4 Embeds 345

http://www.aal-alias.eu 301
https://twitter.com 33
http://aal-alias.eu 8
http://translate.googleusercontent.com 3

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Semantic structuring and linking of event-centric data in the social web Semantic structuring and linking of event-centric data in the social web Presentation Transcript

  • WP4 - semantic structuring andlinking of event-centric datain the social webRaphaël Troncyraphael.troncy@eurecom.fr / @rtroncy
  • WP4: Original WorkplanTask Title DurationT4.1 Specification of an event model for representingpersonal eventsM1 – M32T4.2 Linking personal events to multimedia content fromthe wider worldM1 – M32T4.3 Linking personal events to knowledge and promotingdiverse and opinionated interactionM1 – M32T4.4 Linking with people and discovering new contacts M1 – M32T4.5 Promoting sustainable interaction between users M1 – M3204/06/2013 - ALIAS Final Meeting - Munich, Germany - 2EURECOM: 76 PMTUM-MMK: 4 PM
  • What are Events?Events are observable occurrences grouping04/06/2013 - ALIAS Final Meeting - Munich, GermanyExperiences documented by MediaPeople Places Time- 3
  • http://www.flickr.com/photos/crsan/3697785107
  • User Tasks and ChallengesALIAS Final Meeting - Munich, GermanyGoals: Discover PAST, PRESENT and FUTURE events Live, relive and predict experiences through shared media Enhance search and recommendations mechanisms Support the decision making processChallenges Semantization of Data (events, media, people) Reconciliation of sparse and highly heterogeneous data User-centric design: discovering, deciding and recommending Scalability, Caching and Optimization04/06/2013 - - 5
  • Scenario on e-tickets Supporting the elderly in finding eventsand purchasing tickets online Enabling the elderly to re-live past events they haveattended with their family and relatives Need data about: Events: http://linkedevents.org/ontology/ (LODE) Media: http://www.w3.org/TR/mediaont-10/ Tickets: http://www.heppnetz.de/ontologies/tio/ns.html (TIO) Meteo: http://purl.org/ns/meteo# PersonalizationLocal event guides are more appropriate!04/06/2013 - ALIAS Final Meeting - Munich, Germany - 6
  • Events through the web …04/06/2013 - ALIAS Final Meeting - Munich, GermanyEvent DirectoriesMedia PlatformsSocial NetworksAllows opportunistic/serendipitous discoveryLimited social features and Informationincompleteness (low coverage, weak decision)Aids remembering and sharing past experiencesLack of consistent events descriptionsAttendance: popularity and friends whereabouts,attendees’ opinions.Limited discovery mechanism: need to filter outrelevant information.Explore semantic connections to create better overviews of events- 7
  • There’s a lot of information out there…04/06/2013 - ALIAS Final Meeting - Munich, Germany - 8
  • A lot of information…http://www.flickr.com/photos/mwparenteau/43203978304/06/2013 - ALIAS Final Meeting - Munich, Germany
  • ApproachALIAS Final Meeting - Munich, Germany04/06/2013 - Aggregate heterogeneous data sources Unify into one data model using SemanticWeb technologies Enrich with media, social data andbackground knowledge Identify meaningful and/or interestingrelationships between events, media, andpeople- 10
  • Architecture (generic dashboard)ALIAS Final Meeting - Munich, GermanyReal-time Reconciliation- Tag-based mapping- Instance matchingHeterogeneous data sourcesData CrawlingRDF ConversionRSS UpdateWeb ApplicationsSPARQLREST API (Elda)04/06/2013 - - 11
  • WP4.1: Data (Event) Model04/06/2013 - ALIAS Final Meeting - Munich, Germany Ontology for Events:LODE: http://linkedevents.org/ontology/ Extension for personal events:Based on WP1 outputs (YOU, TUM-GSING, PME)Taxonomy of event categories Triple store gathering:Events and media descriptions: user’s long-termmemoriesSystem interaction: user’s short-term memory- 12
  • ALIAS Final Meeting - Munich, GermanyData Modeling – LODE ontology04/06/2013 -LODE is a minimal model that encapsulates the factual properties ofevents: What, Where, When and Who.- 13
  • Róisín Murphy at Nouveau Casino04/06/2013 - ALIAS Final Meeting - Munich, Germany350591E0-001-005971169-9- 14
  • Representing Events with04/06/2013 - ALIAS Final Meeting - Munich, Germany - 15
  • Róisín Murphy at Nouveau Casino04/06/2013 - ALIAS Final Meeting - Munich, Germany - 16
  • Media explicitly associated with the event04/06/2013 - ALIAS Final Meeting - Munich, Germany2.3+ million imagesover 120.000+ eventsMachine tags“lastfm:events”APIs11.000+ videosover 3000+ events- 17
  • Representing Media with Media Ontology04/06/2013 - ALIAS Final Meeting - Munich, Germany - 18
  • ALIAS Final Meeting - Munich, GermanyData Crawling Various API specifications Policy management Requests chaining Different input and output schemas Time-consumingA Restful Service leveraging the commonalities of API specificationsA Web dashboard to easily handle data collection and interlinkinghttp://eventmedia.eurecom.fr/dashboard/04/06/2013 - - 19
  • Real-Time ReconciliationALIAS Final Meeting - Munich, GermanyParticipantsMediaCost & descriptionPrecise timeagents04/06/2013 -Need for semantic connections between events, media, agents and locations- 20
  • ALIAS Final Meeting - Munich, GermanyLive UpdateTag = *:eventPer week• 1500 Photos• 130 eventsNumber of Items posted per days of the weekNumber of Items posted during the last days04/06/2013 - - 21
  • Interlinking04/06/2013 - ALIAS Final Meeting - Munich, GermanyEventMediaMusicBrainzDBpediaGeonamesFoursquareUberblicFreebaseAgent Last.fm Eventful MusicBrainz DBpedia Freebase Uberblic New York TimesLieu Last.fm Eventful Upcoming DBpedia Freebase Foursquare GeonamesÉvènement Last.fm Eventful Upcoming DBpedia Freebase Uberblic- 22
  • Instance Matching Events/Agents/Locations semantic connectionsSPARQL-based Instance matchingFiltering by the issuance date ChallengesStructural heterogeneityDifferent representations of the same valueUser-generated knowledge featuring human mistakesDifferent granularities between data sources04/06/2013 - ALIAS Final Meeting - Munich, Germany - 23
  •  The events similarity is a mutual agreement of their factual properties Based on top-k dependencies between properties Evaluation: Precision: 96% and Recall: 94% Ground truth of 300 matched instancesALIAS Final Meeting - Munich, GermanyEvents connectionsp1 p2dependencytitle1 title2 0.30place1 place2 0.28title1 agent2 0.26agent1 agent2 0.21description1 title2 0.16Minimal conditions to fetchsimilar events using SPARQL1st levelRefine the results2nd level04/06/2013 - - 24
  •  Agents connections (precision-oriented) Internal reconciliation + DBpedia, Musicbrainz and BBC Label matching + contextual filtering (Cosine) Granularity? (1 to N mappings)  Cosine inequality Locations connections (precision-oriented) Internal reconciliation + DBpedia and Foursquare It is straightforward thanks to the consistency of locations descriptions Top-k Google search Geographical distanceALIAS Final Meeting - Munich, GermanyAgents & Locations connectionsCosine (d, a+b) > Cosine (d ,a)04/06/2013 - - 25
  • About the dataset SPARQL endpoint: http://eventmedia.eurecom.fr/sparql Restful API : http://eventmedia.eurecom.fr/rest/event/ALIAS Final Meeting - Munich, GermanyEvent Agent Location Photos UserLast.fm 61,173 62,577 15,917 1;464,885 130,249Upcoming 19, 276 15 10,269 308,148 3,932Eventful 45,204 8,725 18,763 3,028 321Total 125,650 71,313 44,948 1,760,787 159,263DBpedia MusicBrainz EventfulLast.Fm 13064 34247 2865Eventful 2515 3616 -04/06/2013 - - 26
  • WP4.2: Linking Events with Media Start with media assets explicitly associatedwith some events Enrich the dataset using large scale multimediaanalysis combined with event properties(time, location, topic)Learning, training, re-training: image and speech!Apply to personal media collections and enrich gradually04/06/2013 - ALIAS Final Meeting - Munich, Germany - 27
  • User Uploading Behavior Analysis Most of event-related photos are uploaded infive days.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 28
  • Achievements Devise a generic approach for collecting mediaoriginating from specific events based on theirvisual characteristics, capture date, locationand owner. Propose two alternative approaches forassessing visual similarity between canditatemedia and known event media:Visual matching against known event media (k-NN)Visual event model trained from examples (SVN)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 29
  • The Framework (Visual Matching) The medias are collected via the Flickr API,thanks to the events’ multiple facets Visual Pruning and Owner Refinement areemployed to improve the performance04/06/2013 - ALIAS Final Meeting - Munich, GermanylocationtimetitleidEventquery bymachine tagqueryby geoqueryby textMediaOwnerRefineResultsPruningMediaMedia- 30
  • Media Collection• Multimodal features are exploited forcollecting data:• Location• Time• Text04/06/2013 - ALIAS Final Meeting - Munich, GermanyFlickrAPI- 31
  • Visual Pruning and Owner Refinement04/06/2013 - ALIAS Final Meeting - Munich, Germany - 32
  • Event Enricher results04/06/2013 - ALIAS Final Meeting - Munich, Germany - 33
  • Event Enricher results04/06/2013 - ALIAS Final Meeting - Munich, Germany - 34
  • Automated Event Modeling Framework04/06/2013 - ALIAS Final Meeting - Munich, GermanyEventtagsRank tagsby frequencytag1tag2tag3……….tagNRank Photosby distance to tagsTop NtagsPic1Pic2Pic3PicM ……PositiveSamplesNegativeSamplesTop MPhotosEventModel- 35
  • ResultsEventID Query Our Algorithm k-NN Pruninglastfm:804783 87.92 88.68 46.98lastfm:1830095 74.81 78.38 80.26lastfm:1858887 61.84 63.41 63.56lastfm:1499065 9.47 90.53 89.94lastfm:1787326 0.00 98.40 92.65lastfm:1351984 96.32 96.32 55.32lastfm:1842684 87.28 87.93 67.86lastfm:2020655 99.21 91.80 71.69lastfm:1301748 93.53 93.53 73.73lastfm:1370837 83.73 85.15 73.83ACMMM10 88.56 91.04 87.56SIGIR2010 0.00 60.19 42.28ACMMM07 25.42 57.62 46.61NICECarnival2011 22.30 76.58 59.10Average Accuracy 69.41 83.31 68.6404/06/2013 - ALIAS Final Meeting - Munich, Germany - 36
  • WP4.3: Linking Events with Knowledge Add semantic categories to media assets Re-rank and re-organize media content forpersonalized user groupsLanguage detection Enrich multimedia presentation with richknowledge and content from the Semantic WebOpinions: articles from newspapers (e.g. New York Times)Topical diversity04/06/2013 - ALIAS Final Meeting - Munich, Germany - 37
  • Sharing photos vs event categories04/06/2013 - ALIAS Final Meeting - Munich, Germany - 38
  • Sharing photos vs number of participants04/06/2013 - ALIAS Final Meeting - Munich, Germany - 39
  • Sharing photos vs country of participants04/06/2013 - ALIAS Final Meeting - Munich, Germany - 40
  • WP4.4: People, Social Network and Events User attendance, user profilesPopularity and friends’ whereaboutsSimilar interests, network, recommendations (behaviors) Social networksEventMedia Facebook applicationhttps://www.facebook.com/TheEventMedia(like us!)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 41
  • Co-participating events in San Francisco04/06/2013 - ALIAS Final Meeting - Munich, Germany - 42
  • Towards user profiles (LDA, T = 10)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 43
  • Facebook native event application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 44
  • Facebook native event application Event model is weak:missing rich categories, the performers and artists, etc.the user cannot reuse existing events Task: design a Facebook application thatallows for creating and sharing events, andclosely interacting with EventMedia Task: crawl public events and related datahosted by Facebook using the Graph API inorder to continuously feed EventMedia04/06/2013 - ALIAS Final Meeting - Munich, Germany - 45
  • EventMedia Facebook application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 46https://www.facebook.com/TheEventMedia EventMedia meets here.com Results (worldwide) 4763 venues have been interlinked 30011 events organized in those venues 465175 photos shared on social networksduring events at those venues Examples: http://here.com/services/place/528u173z-672bb28e202e4032848cd6a1fe4b3851(560 events, 9462 photos) oneventmedia http://here.com/services/place/040u29fp-70d8361e99d64a4d978f107eaa69b15a(2 events, 542 photos) on eventmedia
  • 04/06/2013 - ALIAS Final Meeting - Munich, Germany - 47
  • WP4.5: Promoting Sustainable Interaction Developing end-user interfaces for searchingand browsing media contentAllow opportunistic/serendipitous discoveryAllow social feature, notification Support decision making upon attendingevents Module for cross lingual chat or other IMGoogle translation service04/06/2013 - ALIAS Final Meeting - Munich, Germany - 48
  • Web Application04/06/2013 - ALIAS Final Meeting - Munich, Germany - 49
  • Web Application Challenges:Modern single-page app on top of semantic technologiesFast, user-friendly and responsive UIDealing with high variability of data Technologies:Backbone.js: MVC framework enables complex andresponsive UIElda Linked Data API maps SPARQL queriesto RESTful URLs04/06/2013 - ALIAS Final Meeting - Munich, Germany - 50
  • Whats on this evening?ALIAS Final Meeting - Munich, Germany04/06/2013 - - 51
  • Whats on this evening?ALIAS Final Meeting - Munich, Germany04/06/2013 - - 52
  • Demo: Live your eventALIAS Final Meeting - Munich, Germany04/06/2013 -http://eventmedia.eurecom.fr- 53
  • Demo: Parallel Facetted BrowsingALIAS Final Meeting - Munich, Germany04/06/2013 -http://eventmap-ui.appspot.com/- 54
  • EventMedia has been awarded04/06/2013 - ALIAS Final Meeting - Munich, Germany - 55http://www.youtube.com/watch?v=8dVD0SqCB6s
  • ALIAS Chat Module (with translation)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 56
  • WP4: Deliverables (all delivered)04/06/2013 - ALIAS Final Meeting - Munich, GermanyDeliverable Title Due dateD4.1 Module for representing events related to user’s long term andshort term memoryM7 (Jan 2011)D4.2 Module for cross-media linking of personal events to web content,v1M14 (Sep 2011)D4.3 Module for knowledge enrichment of event descriptions M14 (Sep 2011)D4.4 Module for personalized discovery of new contacts on line M30 (Dec 2012)D4.5 Module for retrieval of opinionated content M33 (Mar 2013)D4.6 Module for cross-media linking of personal events to web content,v2M32 (Feb 2013)D4.7 Module for recommendation of topics and activities suitable formaintaining social contactsM32 (Feb 2013)D4.8 Module for cross-lingual chat support M35(May 2013)- 57
  • WP4 Publications and Impact 26 high quality publications + 2 awards + 3 more under review: RecSys 2013 + Journal of Web Semantics +Semantic Web Journal Semantic Web Challenge 2012 at ISWC 2012: 1st price IESD Challenge at EKAW 2012: 1st price MediaEval Benchmark 2011: 7 participants (18 interested) 2012: 5 participants (25 interested) 2013: ongoing 2 PhD Thesis: Xueliang Liu (03/12/2012): Event-based Social Media Data Mining Houda Khrouf (scheduled for the end of 2013)04/06/2013 - ALIAS Final Meeting - Munich, Germany - 58
  • Publications 1/41. Raphaël Troncy, Ryan Shaw and Lynda Hardman. LODE: une ontologie pour représenter desévénements dans le web de données (on HAL). In 21th Journées Francophones dIngénierie desConnaissances (IC10), pages 69-80, Nîmes, France, June 8-11, 2010.2. Raphaël Troncy, Ryan Shaw and Lynda Hardman. LODE: une ontologie pour représenter desévénements dans le web de données (on HAL). In 21th Journées Francophones dIngénierie desConnaissances (IC10), pages 69-80, Nîmes, France, June 8-11, 2010.3. Raphaël Troncy, Bartosz Malocha and André Fialho. Linking Events with Media. In the OpenTrack of the Linked Data Triplification Challenge, colocated with the 6th InternationalConference on Semantic Systems (I-SEMANTICS10), Graz, Austria, September 1-3, 2010.4. Raphaël Troncy, André Fialho, Lynda Hardman and Carsten Saathoff. Experiencing Eventsthrough User-Generated Media. In (ISWC10) International Workshop on Consuming Linked Data(COLD10), CEUR Proceedings Vol. 665, Shanghai, China, November 7-11, 2010.5. Xueliang Liu, Raphaël Troncy and Benoît Huet. Finding Media Illustrating Events. In 1st ACMInternational Conference on Multimedia Retrieval (ICMR11), Trento, Italy, April 17-20, 20116. Houda Khrouf and Raphaël Troncy. Réconcilier les événements dans le web de données. In22nd Journées Francophones dIngénierie des Connaissances (IC11), Chambéry, France, May16-20, 2011.7. Martha Larson, Adam Rae, Claire-Helene Demarty, Christoph Kofler, lorian Metze, RaphaëlTroncy, Vasileios Mezaris, Gareth J.F. Jones. Working Notes Proceedings of the MultimediaBenchmark Workshop (MediaEval 2011), CEUR Proceedings Vol. 807, editors, Santa Croce inFossabanda, Pisa, Italy, September 1-2, 2011.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 59
  • Publications 2/48. Symeon Papadopoulos, Raphaël Troncy, Vasileios Mezaris, Benoît Huet and IoannisKompatsiaris. Social Event Detection at MediaEval 2011: Challenges, Dataset and Evaluation. InMediaEval Benchmarking Initiative for Multimedia Evaluation, CEUR Proceedings Vol. 807, Pisa,Italy, September 1-2, 2011.9. Xueliang Liu, Benoît Huet and Raphaël Troncy. EURECOM @ MediaEval 2011 Social EventDetection Task. In MediaEval Benchmarking Initiative for Multimedia Evaluation, CEURProceedings Vol. 807, Pisa, Italy, September 1-2, 2011.10. Marieke van Erp, Willem Robert van Hage, Laura Hollink, Anthony Jameson and RaphaëlTroncy. Proceedings of the International Workshop on Detection, Representation, andExploitation of Events in the Semantic Web (DeRiVE11), CEUR Proceedings Vol. 779, editors,Bonn, Germany, October 23, 2011.11. Houda Khrouf and Raphaël Troncy. EventMedia Live: Reconciliating Events Descriptions in theWeb of Data. In (ISWC11) 6th International Workshop on Ontology Matching (OM11), Bonn,Germany, October 23-27, 2011.12. Houda Khrouf and Raphaël Troncy. EventMedia: Visualizing Events and Associated Media.Demo Session at the 10th International Semantic Web Conference (ISWC2011), Bonn, Germany,October 23-27, 2011.13. Xueliang Liu, Raphaël Troncy and Benoît Huet. Using Social Media to Identify Events. In (ACMMultimedia) 3rd Workshop on Social Media (WSM11), Scottsdale, Arizona, USA, November 28-December 1st, 2011.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 60
  • Publications 3/414. Liu, Xueliang; Huet, Benoit. Social event discovery by topic inference. In 13th InternationalWorkshop on Image Analysis for Multimedia Interactive Services (WIAMIS 2012), 23-25 May2012, Dublin City University, Ireland15. Houda Khrouf, Ghislain Atemezing, Thomas Steiner, Giuseppe Rizzo and Raphaël Troncy.Confomaton: A conference enhancer with social media from the cloud. In 9th ExtendedSemantic Web Conference (ESWC 2012), May 27-31, 2012, Heraklion, Crete16. Houda Khrouf, Ghislain Atemezing, Giuseppe Rizzo, Raphaël Troncy and Thomas Steiner.Aggregating social media for enhancing conference experience. In 1st International Workshopon Real-Time Analysis and Mining of Social Streams (RAMSS 2012), June 4, 2012, Dublin,Ireland17. Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy and IoannisKompatsiaris. Social Event Detection at MediaEval 2012: Challenges, Dataset and Evaluation. InMediaEval Benchmarking Initiative for Multimedia Evaluation, CEUR Proceedings Vol. 927, Pisa,Italy, October 4-5, 2012.18. Sven Buschbeck, Anthony Jameson, Raphaël Troncy, Houda Khrouf, Osma Suominen andAdrian Spirescu. A Demonstrator for Parallel Faceted Browsing. In (EKAW) IntelligentExploration of Semantic Data Workshop (IESD 2012), Winner of the IESD Challenge, Galway,Ireland, October 8-12, 2012.19. Marieke van Erp, Laura Hollink, Willem Robert van Hage, Raphaël Troncy and David A. Shamma.Proceedings of the Second International Workshop on Detection, Representation, andExploitation of Events in the Semantic Web (DeRiVE 2012), CEUR Proceedings Vol. 902, editors,Boston, USA, November 12, 2012.04/06/2013 - ALIAS Final Meeting - Munich, Germany - 61
  • Publications 4/420. Houda Khrouf, Vuk Milicic and Raphaël Troncy. EventMedia Live: Exploring Events Connectionsin Real-Time to Enhance Content. In 11th International Semantic Web Conference (ISWC 2012),First Prize Winner of the Semantic Web Challenge, Boston, USA, November 11-15, 2012.21. Xueliang Liu and Benoit Huet. Gathering training sample automatically for social event visualmodeling. ACM International Workshop on Socially Aware Multimedia (SAM 2012), inconjunction with ACM Multimedia 2012, 29 October 2012, Nara, Japan22. Symeon Papadopoulos, Emmanouil Schinas, Vasileios Mezaris, Raphaël Troncy and IoannisKompatsiaris. The 2012 Social Event Detection Dataset. In ACM Multimedia Systems (MMSys2013), Dataset Session, Oslo, Norway, February 27-March 1, 201323. Sven Buschbeck, Raphaël Troncy, Anthony Jameson, Houda Khrouf, Adrian Spirescu, OsmaSuominen, Tanja Schneeberger and Eero Hyvönen. Parallel Faceted Browsing. In ACMConference on Human Factors in Computing Systems (CHI 2013), Interactivity Track, Paris,France, April 27-May 2, 201324. Xueliang Liu and Benoit Huet. EventEnricher: a novel way to collect media illustrating events.ICMR 2013, ACM International Conference on Multimedia Retrieval, April 16-20, 2013, Dallas,Texas, USA25. Xueliang Liu and Benoit Huet. Heterogeneous features and model selection for event-basedmedia classification. ICMR 2013, ACM International Conference on Multimedia Retrieval, April16-20, 2013, Dallas, Texas, USA26. Xueliang and Benoit Huet. On the automatic online collection of training data for visual eventmodeling. Multimedia Tools and Applications, February 2013, ISSN: 1380-750104/06/2013 - ALIAS Final Meeting - Munich, Germany - 62
  • http://www.slideshare.net/troncy04/06/2013 - ALIAS Final Meeting - Munich, Germany - 63