Your SlideShare is downloading. ×
Finding media illustrating events
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Finding media illustrating events

38,536
views

Published on

Finding Media Illustrating Events, Talk given at the 1st International Conference on Multimedia Retrieval (ICMR), April 2011, Trento , Italy

Finding Media Illustrating Events, Talk given at the 1st International Conference on Multimedia Retrieval (ICMR), April 2011, Trento , Italy

Published in: Technology

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
38,536
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
20
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Finding Media Illustrating EventsXueliang Liu <xueliang.liu@eurecom.fr>Raphaël Troncy <raphael.troncy@eurecom.fr>Benoit Huet <benoit.huet@eurecom.fr>
  • 2. What are Events? Events are observable occurrences grouping People Places Time Experiences documented by Media 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -2
  • 3. http://www.flickr.com/photos/crsan/3697785107
  • 4. Searching for an event 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -4
  • 5. Searching for media about an event 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -5
  • 6. There’s a lot of information out there… 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -6
  • 7. http://www.flickr.com/photos/mwparenteau/432039783 19/04/2011 - 7
  • 8. Organize the mess Event Media Scrape event directories Link the information Find media illustrating events Design the application Interface Linking Open Data cloud diagram, by Richard Cyganiak and Anja Jentzsch. http://lod-cloud.net/ 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -8
  • 9. Róisín Murphy at Nouveau Casino E0-001-005971169-9 350591 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy -9
  • 10. Representing Events with 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 10
  • 11. Interlinking Linking Agents with  Freebase, Dbpedia, MusicBrainz Linking Venues with  Geonames, Dbpedia, Foursquare (via Uberblic) Linking Events with  Last.fm, Upcoming, Eventful Linking Categories with  Facebook, Eventful, Upcoming, Zevents, LinkedIn,Eventbrite, TicketMaster Linking Users with  Social Graph API 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 11
  • 12. Róisín Murphy at Nouveau Casino 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 12
  • 13. Media explicitly associated with the event APIs Machine tags “lastfm:events” 4790 photos, 263 1.7 million images over videos over 110 events 108.000 events 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 13
  • 14. Representing Media with Media Ontology 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 14
  • 15. How much data is there? Event Agent Location Photos UserLast.fm 57,258 50,150 16,471 1,425,318 18,542Upcoming 13,114 0 7,330 347,959 4,518Eventful 37,647 6,543 14,576 0 0Total 108,019 56,693 38,377 1,773,277 23,060 1,248,021 geo-tagged photos by propagating information from events! 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 15
  • 16. How fast media are uploaded? 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 16
  • 17. Finding more media that illustrate an eventA. Compute the bounding box area of a venueB. Retrieve all media geo-tagged in this areaC. Retrieve all media with a similar titleD. Prune the results with visual analysisE. Extend the result set with all media from the same uploader 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 17
  • 18. A. Bounding box of Nouveau Casino? 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 18
  • 19. B. 74 photos taken in this area this day 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 19
  • 20. C. 85 additional photos with a similar title 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 20
  • 21. D. 6 photos after visual pruning        19/04/2011 -  Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 21 
  • 22. How is the visual pruning performed? Model dataset: photo id + photo geo Testing dataset: similar title Low-level features used:  Color moments, Gabor texture, Edge histogram L1 distance on the K-nearest neighbors Threshold  Min L1 distance between two model image pairs  Conservative approach 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 22
  • 23. E. 66 photos after uploader heuristics  hellerpop DustGraph / Stefan cartoixa 13 photos 46 photos 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 23
  • 24. Same process for videos1 video (id)3 videos (geo) 26 videos (title) Visual pruning performed on key frames   Nb positive > 50% 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 24
  • 25. How illustrated are the events? Query By ID Query By Geo Query By Title Visual Pruning Heuristic Photos 5 74 (74) 85 (85) 6 (6) 66 (66) Videos 1 3 (0) 23 (0) 13 (0) - (title) Videos 10 (10)(title+venue)  20 events  Model dataset: 785 photos  Testing dataset: 1766 photos (1573 positive, 193 negative)  Results: 439 photos (99% precision, 28% recall) 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 25
  • 26. User Interface 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 26
  • 27. Conclusions Method for finding media illustrating scheduled events  Search media with machine and geo tags  Search media with title and normal tags  Prune visually and retrieve all media from confirmed users  Challenge: do not necessarily trust the geo-coordinates Event-based approach for users to explore, annotate and share media  UX can help semantics, semantics can help UX  Outstanding challenges in interlinking and curating the data  Detecting events from social media activity 19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 27
  • 28. http://www.slideshare.net/troncy19/04/2011 - Finding Media Illustrating Events - ICMR 2011, Trento, Italy - 28