Looking for a Needle in Video Haystack #appsummit14

505 views
380 views

Published on

As the fastest-growing type of content on the Internet, consumer produced videos are a wealth of information about the world that's essentially untapped. We present ICSI's research on the large-scale video search methods using an application that reveals the geo-location of a consumer- produced video based on its content. Gerald Friedland, University of California, Berkeley.

Published in: Technology, Business
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
505
On SlideShare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
3
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Looking for a Needle in Video Haystack #appsummit14

  1. 1. Finding the Needle 
 in the Video Haystack Dr. Gerald Friedland Director Audio and Multimedia Research International Computer Science Institute Berkeley, CA friedland@icsi.berkeley.edu
  2. 2. The Internet is Multimedia 2
  3. 3. Multimedia in the Internet is Growing 3
  4. 4. Multimedia in the Internet is Growing 3 • YouTube alone claims 48 72 100 hours video uploads every minute.
  5. 5. Multimedia in the Internet is Growing 3 • YouTube alone claims 48 72 100 hours video uploads every minute. • Youku (Chinese YouTube) claims 80k video uploads per day
  6. 6. Multimedia in the Internet is Growing 3 • YouTube alone claims 48 72 100 hours video uploads every minute. • Youku (Chinese YouTube) claims 80k video uploads per day • Flickr, Instagram, Liveleak, Vimeo...
  7. 7. 4
  8. 8. The Opportunity 5
  9. 9. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: 5
  10. 10. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, 5
  11. 11. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, – medicine, 5
  12. 12. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, – medicine, – economics, 5
  13. 13. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, – medicine, – economics, – … 5
  14. 14. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, – medicine, – economics, – … • Problem: Videos need to be searchable beyond keywords. 5
  15. 15. The Opportunity • Consumer-Produced Multimedia allows empirical studies at never-before-seen scale: – sociology, – medicine, – economics, – … • Problem: Videos need to be searchable beyond keywords.• 5
  16. 16. Our Approach 6 Ball sound Male voice (near) Child’s voice (distant) Child’s whoop (distant) Room tone Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
  17. 17. Our Approach Multimodal exploitation of video content, including audio and temporal information.6 Ball sound Male voice (near) Child’s voice (distant) Child’s whoop (distant) Room tone Cameron learns to catch (http://www.youtube.com/watch?v=o6QXcP3Xvus)
  18. 18. Location Estimation 7 J. Choi, G. Friedland, V. Ekambaram, K. Ramchandran: "Multimodal Location Estimation of Consumer Media: Dealing with Sparse Training Data," in Proceedings of IEEE ICME 2012, Melbourne, Australia, July 2012.
  19. 19. Bayesian graphical framework 8 {berkeley,  sathergate,   campanile} {berkeley,  haas} {campanile} {campanile,  haas} Node:  Geoloca7on  of   the  image Edge:  Correlated  loca7ons   (e.g.  common  tag) Edge  Poten,al:  Strength  of  an  edge,   (e.g.  posterior  distribu7on  of  loca7ons   given  common  tags) p(xi, xj|{tk i } {tk j }) p(xj|{tk j })p(xi|{tk i })

×