Arcomem training multimedia

320 views

Published on

This presentation on understanding Images and Video detection is part of the ARCOMEM training curriculum. Feel free to roam around or contact us on Twitter via @arcomem to learn more about ARCOMEM training on archiving Social Media.

Published in: Technology, Art & Photos
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
320
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
10
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Arcomem training multimedia

  1. 1. ARCOMEM Training Material Understanding Images and Video
  2. 2. ARCOMEM Training Material Introduction • The web is increasingly multimedia in nature • We concentrate on the multimedia aspects • Image and video content analysis, indexing, matching and annotation • Exploit the web, social web and linked data web
  3. 3. ARCOMEM Training Material Exploring image and video reuse on the Web
  4. 4. ARCOMEM Training Material • Goal – Use image analysis techniques to aggregate social contexts; in particular to interlink the social discussion of events and topics through the content of images and video. • Motivation – Media is often reused or reposted on social networks. – Detection of near-duplicate multimedia artifacts provides a means to investigate and explore many facets about the documents the media is embedded within. • Some examples include: – Aggregating documents about the same subject/event/opinion » Finding cases where media is used in differing contexts is also interesting. – Exploring how different social groups talk about the same media
  5. 5. ARCOMEM Training Material
  6. 6. ARCOMEM Training Material ARCOMEM US-Elections Crawl Examples
  7. 7. ARCOMEM Training Material Images • image/gif 4371 • image/jpeg 667073 • image/png 46644 • image/svg+xml 1 • image/x-icon 2 • image/x-ms-bmp 167
  8. 8. ARCOMEM Training Material 122 Detected Duplicates https://sphotos-a.xx.fbcdn.net/hphotos-ash3/63089_10151299959267177_1113898282_n.jpg http://p.twimg.com/A7Oq4JRCEAA05uG.jpg https://p.twimg.com/A7RMKgrCIAAvPdj.jpg http://24.media.tumblr.com/tumblr_md4ttulK8d1qa5ex8o1_500.jpg https://p.twimg.com/A7Oq4JRCEAA05uG.jpg https://p.twimg.com/A7JYdoDCEAAktAb.jpg https://p.twimg.com/A7JIdp7CcAAjsPF.jpg http://p.twimg.com/A7JSUYCCUAEzW5O.jpg http://p.twimg.com/A7NoTNOCIAEQ3ie.jpg https://p.twimg.com/A7LsnVrCQAAWYFA.jpg ... 112 Detected Duplicates http://25.media.tumblr.com/tumblr_mc0mmx2kTA1rxxq3ro1_500.png http://p.twimg.com/A7Ep0CjCMAAon-f.jpg http://p.twimg.com/A7N3lxrCAAE_gzP.jpg https://p.twimg.com/A5ctYtVCMAE8_8e.jpg http://p.twimg.com/A7NMvVACUAEL9aa.jpg http://p.twimg.com/A7EkZK8CEAE0pYu.jpg https://p.twimg.com/A7FOSwtCUAADws1.jpg http://p.twimg.com/A7EiFd4CcAAZdT1.jpg http://p.twimg.com/A7JwHHbCQAAvqNN.jpg ...
  9. 9. ARCOMEM Training Material 103 Detected Duplicates https://p.twimg.com/A7K8WWiCAAE9Uap.png https://p.twimg.com/A7Iq1DSCQAAyRBG.jpg http://p.twimg.com/A7L7bQJCIAIaaqm.jpg http://p.twimg.com/A7KSC5gCcAAcLTj.jpg https://p.twimg.com/A7OqzarCYAA8L9N.jpg https://p.twimg.com/A7OlKn-CQAA-j7X.jpg https://p.twimg.com/A7J5CHPCUAEp-86.png https://p.twimg.com/A7JOd9kCcAIy3YM.jpg http://25.media.tumblr.com/tumblr_lyr1v4kp8J1qcjsjlo1_500.jpg ... 106 Detected Duplicates https://p.twimg.com/A7MQiKeCMAE1rmX.jpg https://p.twimg.com/A7IFNt_CUAAqYFV.jpg https://p.twimg.com/A7DrPLMCIAAoKb8.jpg http://p.twimg.com/A7Y5BW9CQAAqF3E.jpg https://p.twimg.com/A2YzbudCQAE77pr.jpg https://p.twimg.com/A7FQF7PCUAAgWWH.jpg https://p.twimg.com/A7EO5YFCMAAMTxB.jpg http://p.twimg.com/A7IWHTtCUAIAtH8.jpg http://p.twimg.com/A7FcyZdCUAAO_Vg.jpg http://25.media.tumblr.com/tumblr_m7c9nivzwF1qfep67o1_500.jpg ...
  10. 10. ARCOMEM Training Material
  11. 11. ARCOMEM Training Material Multimedia Opinion Mining
  12. 12. ARCOMEM Training Material Goals • Investigate the use of facial analysis to classify facial expressions in images and videos found on the web. – Can be used to indicate emotion of subject. • Investigate course-grained automatic classification using image features – For abstract opinion-related concepts (sentiment/privacy/attractiveness) • Investigate correlations between images and opinions mined from text – Does the same image get reused in different documents to illustrate the same (or different) opinion?
  13. 13. ARCOMEM Training Material Sentiment/privacy/attractiveness • Experimental support for classifying visual media with respect to sentiment, privacy and attractiveness is being built into ARCOMEM.
  14. 14. ARCOMEM Training Material Image-opinion correlation • Correlations between images and opinions extracted from the text can be explored by querying the ARCOMEM database. +ve -ve
  15. 15. ARCOMEM Training Material Towards Facial Analysis in the Wild • Detection and analysis of faces in multimedia content can help us guide and contextualise a crawl: – Recognition and expression analysis can help us determine if an image is relevant or interesting. – Post-crawl the information can be used for visualisation. • Current research very much based on images taken in lab-conditions; how far can we take it?
  16. 16. ARCOMEM Training Material Applying the model to crawled images

×