The document discusses an enhanced multimedia search algorithm called "Time series Active Search Algorithm" that uses feature vectors and histogram pruning to enable faster and more efficient searching of audio and video queries from large stored datasets. The algorithm extracts feature vectors from query and stored signals, classifies the vectors, obtains histograms, compares histograms using a threshold, and notifies of similarities. Experimental results found the algorithm reduced matches to over 1/291 for video and 1/501 for audio, making search very fast and reasonably robust even with noise and compression.

1. ENHANCED MULTIMEDIA SEARCH Lakshman Prasad 4MC02EC032 VIII semester E&C

2. Multimedia Search Types: Multimedia Search Content based Indexing, classifying ‘ Widely used today- Google Human’ factor Efficient when accurately indexed Similarity based Correlation of data / feature vectors Infallible Scope of usage very high Efficient for multimedia search

3. Similarity Based Search Search  Similarity based Use of ‘raw image data’ correlation Direct correlation of binary data Efficient for exact matching Use of extracted ‘feature vectors’ Extraction of ‘feature vectors’ Efficient when approximate matching suffices

4. ‘ Feature Vectors’ Search  Similarity based  Feature vectors Full histogram comparison Direct comparison of histograms Simpler implementation Inefficient for large data. Use of ‘histogram pruning’ Comparison of parts of histogram Complex implementation Efficient search for large data

5. ‘ Enhanced Multimedia Search =’ Search  Similarity based  Feature vectors  Histogram Pruning Search Similarity based search Feature vectors extracted similarity based search “ Histogram pruning based feature vector extracted similarity based search” “ Time series Active Search Algorithm”

6. Algorithm Apply windows of query signal length Extract feature vectors from query signal Extract feature vectors from stored signal Classify feature vectors Obtain histograms Determine threshold of comparison Compare histograms Notify when similar / accurate Otherwise, shift forward the window, prun Obtain next Histogram, repeat

7. Time Series Active Search

8. Feature vector used Audio- Short term f spectrum thro’ BPF where k is the sampled time, f j (k) is the normalized short time power spectrum by, where y j (t) is output WF of BPF j at t. M is the time support of feature vector N is the # of frequency channels Video- Color distribution

9. Implementation details Histogram Similarity Pruning Detection criterion

10. Histogram Pruning Analogy EXAMPLE H ERE IS A SIMPLE EXAMPLE E XAMPLE H E RE IS A SIMPLE EXAMPLE E XAMPLE H E R E IS A SIMPLE EXAMPLE E XAMPLE HER E IS A SIMPLE EXAMPLE E XAMPLE HER E IS A SIMPLE EXAMPLE E XAMPLE HERE I S A SIMPLE EXAMPLE … ...... E XAMPLE HERE IS A SIMPLE E XAMPLE EX AMPLE HERE IS A SIMPLE E X AMPLE ……… EXAMPLE HERE IS A SIMPLE EXAMPLE

11. ‘ AND’ search

12. Experimental results: Speed 48h stored, 15s query PIII 933 MHz PC, Linux Audio: VHS HI-Fi, 11kHz sampling f, 16 bit Q, 7ch II order IIR BPF, M=110 Video: NTSC, 29.97 fps, 160x120 pixels/image, W=16

13. Experimental results: Accuracy 60m TV broadcast used as query & stored signal Random choice 100 times repeated Gaussian noise for audio MPEG1 compression for video

14. Experimental results: Summary Matches reduced to 1/291 (video) and , 1/501 (audio), very fast Accurate and reasonable robust with AWGN and MPEG1 compression

15. Extensions to Algorithm ‘ OR’ model of parallel search Feature Distortion Absorption model Dynamic Threshold value determination

16. Applications Statistical analysis of broadcasted music, TV / radio commercials Copyright management on Internet ‘ Truly similar search’- similar music Fastening of biometric security algorithms

17. Conclusion Discussed algorithm enables a faster and more efficient search of audio and video queries from long running audio/video More efficient ‘AND’ method discussed, other extensions overviewed Web search engines based on audio and video queries anticipated

18. References “ A quick search method for Audio and Video signals based on histogram pruning” IEEE transactions on multimedia, Vol.5, #3, Sept 2003 Other IEEE transactions on Multimedia “ Digital Image Processing” Rafael C Gonzalez, Richard E woods “ Digital Signal Processing” Proakis and Manolakis http://www-igm.univ-mlv.fr/~lecroq/string/node14.html http://ma.i2r.a-star.edu.sg/Staff/~lingyu/FastVSearch.pdf http://www.cs.utexas.edu/users/moore/best-ideas/string-searching/kpm-example.html http://search.aol.co.uk/av_idx?help=av_adv Several links from http://www.google.com/search?hl=en&q=audio+and+video+search&btnG=Google+Search