Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Seminar

1,058 views

Published on

Enhanced Mutimedia Search

Published in: Technology, News & Politics
  • Be the first to comment

  • Be the first to like this

Seminar

  1. 1. ENHANCED MULTIMEDIA SEARCH Lakshman Prasad 4MC02EC032 VIII semester E&C
  2. 2. Multimedia Search Types: Multimedia Search <ul><li>Content based </li></ul><ul><ul><li>Indexing, classifying </li></ul></ul><ul><ul><li>‘ Widely used today- Google </li></ul></ul><ul><ul><li>Human’ factor </li></ul></ul><ul><ul><li>Efficient when accurately indexed </li></ul></ul><ul><li>Similarity based </li></ul><ul><ul><li>Correlation of data / feature vectors </li></ul></ul><ul><ul><li>Infallible </li></ul></ul><ul><ul><li>Scope of usage very high </li></ul></ul><ul><ul><li>Efficient for multimedia search </li></ul></ul>
  3. 3. Similarity Based Search Search  Similarity based <ul><li>Use of ‘raw image data’ correlation </li></ul><ul><ul><li>Direct correlation of binary data </li></ul></ul><ul><ul><li>Efficient for exact matching </li></ul></ul><ul><li>Use of extracted ‘feature vectors’ </li></ul><ul><ul><li>Extraction of ‘feature vectors’ </li></ul></ul><ul><ul><li>Efficient when approximate matching suffices </li></ul></ul>
  4. 4. ‘ Feature Vectors’ Search  Similarity based  Feature vectors <ul><li>Full histogram comparison </li></ul><ul><ul><li>Direct comparison of histograms </li></ul></ul><ul><ul><li>Simpler implementation </li></ul></ul><ul><ul><li>Inefficient for large data. </li></ul></ul><ul><li>Use of ‘histogram pruning’ </li></ul><ul><ul><li>Comparison of parts of histogram </li></ul></ul><ul><ul><li>Complex implementation </li></ul></ul><ul><ul><li>Efficient search for large data </li></ul></ul>
  5. 5. ‘ Enhanced Multimedia Search =’ Search  Similarity based  Feature vectors  Histogram Pruning <ul><li>Search </li></ul><ul><ul><li>Similarity based search </li></ul></ul><ul><ul><ul><li>Feature vectors extracted similarity based search </li></ul></ul></ul><ul><ul><ul><ul><li>“ Histogram pruning based feature vector extracted similarity based search” </li></ul></ul></ul></ul><ul><li>“ Time series Active Search Algorithm” </li></ul>
  6. 6. Algorithm <ul><li>Apply windows of query signal length </li></ul><ul><li>Extract feature vectors from query signal </li></ul><ul><li>Extract feature vectors from stored signal </li></ul><ul><li>Classify feature vectors </li></ul><ul><li>Obtain histograms </li></ul><ul><li>Determine threshold of comparison </li></ul><ul><li>Compare histograms </li></ul><ul><li>Notify when similar / accurate </li></ul><ul><li>Otherwise, shift forward the window, prun </li></ul><ul><li>Obtain next Histogram, repeat </li></ul>
  7. 7. Time Series Active Search
  8. 8. Feature vector used <ul><li>Audio- Short term f spectrum thro’ BPF </li></ul><ul><ul><li>where k is the sampled time, </li></ul></ul><ul><ul><li>f j (k) is the normalized short time power spectrum by, </li></ul></ul><ul><ul><li>where y j (t) is output WF of BPF j at t. </li></ul></ul><ul><ul><li>M is the time support of feature vector </li></ul></ul><ul><ul><li>N is the # of frequency channels </li></ul></ul><ul><li>Video- Color distribution </li></ul>
  9. 9. Implementation details <ul><li>Histogram </li></ul><ul><li>Similarity </li></ul><ul><li>Pruning </li></ul><ul><li>Detection criterion </li></ul>
  10. 10. Histogram Pruning Analogy <ul><li>EXAMPLE </li></ul><ul><li>H ERE IS A SIMPLE EXAMPLE </li></ul><ul><li>E XAMPLE </li></ul><ul><li>H E RE IS A SIMPLE EXAMPLE </li></ul><ul><li>E XAMPLE </li></ul><ul><li>H E R E IS A SIMPLE EXAMPLE </li></ul><ul><li>E XAMPLE </li></ul><ul><li>HER E IS A SIMPLE EXAMPLE </li></ul><ul><li>E XAMPLE </li></ul><ul><li>HER E IS A SIMPLE EXAMPLE </li></ul><ul><li>E XAMPLE </li></ul><ul><li>HERE I S A SIMPLE EXAMPLE </li></ul><ul><li>… ...... </li></ul><ul><li>E XAMPLE </li></ul><ul><li>HERE IS A SIMPLE E XAMPLE </li></ul><ul><li>EX AMPLE </li></ul><ul><li>HERE IS A SIMPLE E X AMPLE </li></ul><ul><li>……… </li></ul><ul><li>EXAMPLE </li></ul><ul><li>HERE IS A SIMPLE EXAMPLE </li></ul>
  11. 11. ‘ AND’ search
  12. 12. Experimental results: Speed <ul><li>48h stored, 15s query </li></ul><ul><li>PIII 933 MHz PC, Linux </li></ul><ul><li>Audio: VHS HI-Fi, 11kHz sampling f, 16 bit Q, 7ch II order IIR BPF, M=110 </li></ul><ul><li>Video: NTSC, 29.97 fps, 160x120 pixels/image, W=16 </li></ul>
  13. 13. Experimental results: Accuracy <ul><li>60m TV broadcast used as query & stored signal </li></ul><ul><li>Random choice </li></ul><ul><li>100 times repeated </li></ul><ul><li>Gaussian noise for audio </li></ul><ul><li>MPEG1 compression for video </li></ul>
  14. 14. Experimental results: Summary <ul><li>Matches reduced to 1/291 (video) and , 1/501 (audio), very fast </li></ul><ul><li>Accurate and reasonable robust with AWGN and MPEG1 compression </li></ul>
  15. 15. Extensions to Algorithm <ul><li>‘ OR’ model of parallel search </li></ul><ul><li>Feature Distortion Absorption model </li></ul><ul><li>Dynamic Threshold value determination </li></ul>
  16. 16. Applications <ul><li>Statistical analysis of broadcasted music, TV / radio commercials </li></ul><ul><li>Copyright management on Internet </li></ul><ul><li>‘ Truly similar search’- similar music </li></ul><ul><li>Fastening of biometric security algorithms </li></ul>
  17. 17. Conclusion <ul><li>Discussed algorithm enables a faster and more efficient search of audio and video queries from long running audio/video </li></ul><ul><li>More efficient ‘AND’ method discussed, other extensions overviewed </li></ul><ul><li>Web search engines based on audio and video queries anticipated </li></ul>
  18. 18. References <ul><li>“ A quick search method for Audio and Video signals based on histogram pruning” IEEE transactions on multimedia, Vol.5, #3, Sept 2003 </li></ul><ul><li>Other IEEE transactions on Multimedia </li></ul><ul><li>“ Digital Image Processing” Rafael C Gonzalez, Richard E woods </li></ul><ul><li>“ Digital Signal Processing” Proakis and Manolakis </li></ul><ul><li>http://www-igm.univ-mlv.fr/~lecroq/string/node14.html </li></ul><ul><li>http://ma.i2r.a-star.edu.sg/Staff/~lingyu/FastVSearch.pdf </li></ul><ul><li>http://www.cs.utexas.edu/users/moore/best-ideas/string-searching/kpm-example.html </li></ul><ul><li>http://search.aol.co.uk/av_idx?help=av_adv </li></ul><ul><li>Several links from </li></ul><ul><li>http://www.google.com/search?hl=en&q=audio+and+video+search&btnG=Google+Search </li></ul>

×