Entropy features trained support vector machine based logo


Published on

  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Entropy features trained support vector machine based logo

  1. 1. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME20ENTROPY FEATURES TRAINED SUPPORT VECTOR MACHINEBASED LOGO DETECTION METHOD FOR REPLAY DETECTIONAND EXTRACTION FROM SPORTS VIDEOSVilas Naik1, Raghavendra Havin21 Department of CSE, Basaveshwar Engineering College, Bagalkot, India2 Department of CSE, Basaveshwar Engineering College, Bagalkot, IndiaABSTRACTIn many sports, the majority of highlights are confined to relatively short durations ofintense action. In some sense these segments capture the essence of a game and summarizethe moments of important action. Automatic detection of these highlights could provide animportant browsing mechanism in a video library of sports games. These replays often cor-respond to highlights in a game and can be used as indices of a sports video. The proposedmechanism employs support vector machine (SVM) for detection of logos that are flashed atbeginning and end of every reply action. The algorithm is composed of logos detection andreplay segment extraction. First SVM is trained with features of all possible logos normallyused in various sports videos. Then the SVM is used for detection of logos that sandwichreplay segment further that segment is automatically extracted to produce replay clip. TheSVM classifier is trained with histogram features of logos is utilized. Experiments conductedon IPL and Soccer videos demonstrate the effectiveness of this method. Moreover, algorithmcan be easily applied to other sports videos where replay is sandwiched by pair of logos.Keywords: Logo-based detection, Support vector machine (SVM learning (ML), Sportsreplay detection, Video summarization.1. INTRODUCTIONIn a sports video library, one may find thousands of hours of recordings, representingmany hundreds of individual games. Fortunately, in any given game, not every minute com-mands the same interest. A majority of the excitement is contained in a small fraction of thecoverage of the game. These highlights are the essence of the game and present a succinctINTERNATIONAL JOURNAL OF GRAPHICS ANDMULTIMEDIA (IJGM)ISSN 0976 - 6448 (Print)ISSN 0976 -6456 (Online)Volume 4, Issue 1, January - April 2013, pp. 20-30© IAEME: www.iaeme.com/ijgm.aspJournal Impact Factor (2013): 4.1089 (Calculated by GISI)www.jifactor.comIJGM© I A E M E
  2. 2. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME21summary of the significant actions and events. For many viewers of sports video these high-lights are all they are really interested in seeing. Detection of these highlights is therefore ofinterest in itself, but might also serve as an important indexing mechanism for searching formore substantial sections of material from large archives. With the rapid increase of mediadata, it is in urgent need of an efficient and effective method for information management andretrieval. Semantics based event detection and indexing has been attracted much attentionrecently for interesting events are highly useful for video browsing, indexing and highlightsgeneration. Replay is an important video editing way in broadcasting programs. Commonly,an important or interesting segment will be played with a slow-motion pattern in order to letaudiences explore and enjoy the detail. Replay is a significant indication for highlights andoften taken as a key factor in event detection. The sports video summarization is widely in-vestigated since sports holds a large amounts of audiences worldwide. As an important com-piled clue, the replay shows the details of an important video segment with a slower speed.Thus it is widely employed in sports video analysis especially for event detection, highlightcreation and summarization etc.Replays are related to the sports, which are shown more than once when any impor-tant event in sports takes place. To get more information from replay, generally it is shownwith slow motion. Hence it is also called slow motion replay (SMR). Replay is which is ne-cessary for highlights generation. Highlight is a shorter version of sports video that is alsocalled as sports video summarization. Replay is important for all spectators and referees tojudge and take final decision in most of the sports. When an important event happen like goalin football or hockey cricket. When referee is in doubtful condition, or strong appeal is madeby players, putting all such events in an original temporal order we can have highlights of anysports video. Replay contains all-important events. Replays are used for highlight making.Replays can be used as a summary of game for searching it in digital video library.Replay is an important video editing way in broadcasting programs. Commonly, animportant or interesting segment will be played with a slow-motion pattern in order to let au-diences explore and enjoy the detail. Replay is a significant indication for highlights and of-ten taken as a key factor in event detection.In recent years, researchers have reported many approaches of replay detection in lite-ratures. In this paper, an automatic and effective logo based replay detection method is pro-posed. It is well known that in most cases there exists a special transition at both start and endof a replay, in which a highlighted logo comes in and out gradually. This transition is calledas “logo-transition”. Logos often keep the same during the whole program. Our motivation isto use the logos to assist replay segments detection. Firstly some logo-transitions are de-tected and further extract logo-samples from them at the beginning of a program. After that,the logo-template is extracted from these samples. Further, this template is employed todetect the other logos. After all logos are obtained, the video can be partitioned into segmentswith taking logos as boundaries. Then, shot and motion features are extracted in each seg-ment. Finally, based on these cues, an SVM classifier is used to replay segment identification.2. RELATED WORKSlow motion replays in broadcast sports video are important and reliable clues forhighlights and key events, since they are usually chosen by human experts. Therefore, detec-tion of replays could facilitate semantic-based high level video processing tasks such as high-light generation [1], event detection [2], video summarization [3], etc.
  3. 3. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME22Some slow motion replay detection methods have been proposed in the literature.They can be broadly classified into three categories. The first category of method directlyanalyzes the inherent attributes of replay video segments and tries to use those attributes todifferentiate the slow motion replays from the normal plays. Before high speed cameras arewidely used, slow motion replays were usually produced by repeating frames recorded fromstandard cameras. A few detection methods are based on this assumption. A replay detectionmethod based on the macroblock type, motion vector and bit rate information is presentd in[4]. The detection is performed in the MPEG compressed domain to find the locations of re-peating frames. The algorithm in [1] calculates the image differences between adjacentframes and search fluctuations in the differences to identify replays. These methods usuallyget low performance on replays that captured by high speed cameras and they can not provideaccurate boundaries of replays.The second category of method employs statistical techniques to perform shot classi-fication for the purpose of detecting replay shots. The author of [5] presented a method basedon the difference of motions between replays and normal plays. A support vector machine(SVM) was trained with features of color ratio, shot length, mean color value and motion re-lated feature to detect replay shots. The proposal in. [6] applied scene transition structureanalysis on the shot classification results. Replays were extracted based on the generated shotlabel sequences by using some predefined rules. A HMM to classify replay and non-replayshots is employed in [7]. The performance of this category is usually not satisfied and resultsare still not precise enough.The third category of method tries to find the differences between replays and normalplays in terms of specific production behaviors, such as special video effects or logo patterns.Replays are then determined according to the existence of such behaviors. The work pre-sented in [8] captures a digital video effect (DVE) l interactively for given videos. It is de-scribed by color and motion of the gradually changing boundary between adjacent shots. Areplays are located by two DVEs detected with the model. Based on the observation thatmore and more replays that are sandwiched in between two certain logo transition sequences,some researchers try to use logo transition locations to identify replays. The mean-square dif-ferences of intensity and color histogram as frame similarity measure to detect logo templateand to search logos in videos is discussed in[9]. A method in [10] employed the spatiotem-poral mode seeking on selected logo sequences to capture the dominant color mode. Thiscolor mode is then used to perform EMD-based similarity matching for logo searching pur-pose. Tong et al. [11] used frame-to-frame difference with MSD to select candidate logo se-quence. They calculated the average image of those candidate logo frames near to the centerto obtain logo template. Template matching was performed based on color and shape fea-tures. Huang et al. [12] first extracted gradual transitions and employed motion features tolearn the logo template. A color representation was then computed from the logo templateand was used to search logos. Dang et al. [13] semiautomatically extracted logo template se-quences and detected replays based on template sequence matching. Han et al. [14] learnedgradual transition patterns based on motion vector reliability classification from a database bySVM. Replay detection was performed by fusing the pattern matching results and slow mo-tion detection results based on motion vector information. The third category of method canusually obtain high accuracy on replay boundaries and the performance is better than abovetwo categories. However, these methods often make assumptions on the logo transition se-quences such as logo locations, moving patterns and speed, types of digital effects, etc. And
  4. 4. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME23nearly all of them assume that the two logo transition sequences before and after the replayare identical.The Proposed solution is a methodology of determining the replay detection in asports video using support vector machine for classifying frames as Logo frame and non logoframe. In most of sports video a logo swaps at beginning and end of the replay, the frames in-between the start logo frame to its corresponding end logo frame of a videos construct a rep-lay event. Applying same procedure for total video it results into collection of replays ofsports video. These replays can be used to form a summary of video.3. PROPOSED ALGORITHM FOR REPLAY DETECTIONThe proposed method employs a Support Vector Machine based classifier for detect-ing the logos that will swap across the screen in broadcasted video at start and end of the rep-lay action.The proposed algorithm consist of mainly TWO stagesi. Train the SVM classifier with logo patterns and non logo patterns for classifyingthe frames in the video as logo frames and non logo frames.ii. Detecting logos at start and end of the replay action and extracting the frames be-tween them.The flow charts for two phases of proposed algorithm are presented in Fig. 1.Fig. 1 Flow diagram of proposed Algorithm, (a) Flow diagram for Training SVM Clas-sifier (b) Flow diagram for Replay Detection and Extraction.Train the SVM Classifier with EntropyFeatures of Logo frame and Non logoframeStartExtract Logo frames from the InputVideo and take some Non logo framesExtract the Entropy Features of aframeENDRead the Input VideoExtract the Entropy Features of eachframesExtraction of frames between twologosTest the frame on trained SVM Clas-sifierDetection of Two logos at beginningand end of each replay eventStartEND
  5. 5. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME24In most of the sports video typical logo-transition lasts 0.5-0.8 seconds or 5-24 frames. Inlogo-transition, there is an image frame that contains the clearest and the most complete logo.Usually, the logo is highlighted and located at the middle part of a frame. With these priorknowledge, the logo-samples from video can be detected and extracted.3.1 Logo-transition DetectionA typical logo-transition lasts 0.5-0.8 seconds or 15-24 frames. The frame-to-framedifference is measured by intensity mean square difference (MSD). The difference sequenceafter median filtering is a plateau-like shaped pattern (Fig. 3). Therefore, the logo-transitiondetection can be performed through plateau-like shape detection over the frame-to-frame dif-ference sequence. The proposed algorithm isi. Compute frame-to-frame difference with MSD.ii. Check the frame difference. Proceed to step 3 When the difference exceeds athreshold, otherwise go back to step 1) for next frame.iii. Count the number of consecutive frame-differences that all exceed the frame-difference-threshold until encounter several consecutive frame-differences thatdrop below this threshold. If the counter exceeds a certain threshold, a wipetransition can be determined. Otherwise go back to step 1) for next frame.Fig. 2 A logo-transition (8 frames are displayed)Fig. 3 Frame difference sequences.Original(left) and after median filtering(right)3.2 Logo-sample DeterminationIn a logo-transition, there is an image frame that contains the clearest and the mostcomplete logo. Usually, the logo is highlighted and located at the middle part of a frame.With these prior knowledge, the logo-samples can be extracted.
  6. 6. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME253.3 Feature ExtractionThe color, texture, shape and motion estimation have been commonly used features inthe literature, It is observed that color and motion features play a dominant role in the extrac-tion of characteristics from the videos and hence the color information is adopted in thiswork. Normally color distribution is estimated in the form of histogram, the proposed workestimates the information of color content at different color intensity planes of images interms of entropy values. The entropy computed for a color plane in an image gives the aver-age information conveyed by an image. In the pr oposed work the entropy of three layers ofRGB are computed and feature vector is written as in equation (1)(1)Where: Feature vector of frame from video sequence: Entropy of Red plane: Entropy of green plane: Entropy of blue planeof an RGB frame from video sequence.The feature vectors as above are constructed for Logo templates extracted from vari-ous videos and some Non logo frames and are used for training SVM.3.4 Support Vector Machine (SVM)The support vector machine (SVM) algorithm seeks to maximize the margin around ahyperplane that separates a positive class from a negative class [20]. Given a training datasetwith n samples (x1, y1); (x2,y2)……….(xn, yn), where xi is a feature vector in a v-dimensional feature space and with labels yi {-1,1}belonging to either of two linearly se-parable classes C1 and C2. Geometrically, the SVM modeling algorithm finds an optimalhyper plane with the maximal margin to separate two classes, which requires to solve the op-timization problem, as shown in equations (2) and (3).where, αi is the weight assigned to the training sample xi. If αi > 0, xi is called a sup-port vector. C is a regulation parameter used to balance the training accuracy and the modelcomplexity so that a superior generalization capability can be achieved. K is a kernel func-tion, which is used to measure the similarity between two samples. Different choices of ker-nel functions have been proposed and extensively used in the past and the most popular arethe gaussian radial basis function (RBF), polynomial of a given degree, and multi layer per-ceptron. These kernels are in general used, independently of the problem, for both discreteand continuous data.(2)(3)
  7. 7. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME263.5 SVM classifier based Replay detection AlgorithmThe proposed algorithm model uses priorly retrieved logos of sports video for thetraining, then its features i.e its entropy values are extracted and trained to SVM classifier.SVM is used as a classifier due to its clear classification method. Some randomly selectedframes without logo are also trained to SVM classifier as Non Logo templates.It classify twoClass vectors by a linear hyper plane , by which classifier can easily classify Logo and NonLogo frames.Fig. 4 Flow chart of replay detection AlgorithmREAD Video file (.avi,.mpeg) frame wiseLOGO_COUNT =0StartExtract all the frames between CurrentLOGO detected and previous detectionwrite them to file which gives a replayevent.Is LOGOframe detected‫ܧ‬௙೔ൌ ሾ ‫ܧ‬௥ ‫ܧ‬௚ ‫ܧ‬௕ሿExtract Entropy features of current frameTest the Frame with trained SVM forLOGO or NON LOGO frameFor LOGO frame LOGO_COUNT =1 forNON LOGO LOGO_COUNT =2End
  8. 8. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME27Algorithm:Step 1: Extract the Logo frames from the input videos as explained in above sections.Step 2: Extract the entropy features of a frames as given in section 3.3.Step 3: Store these entropy features into vectors as feature patterns.Step 4: Train the SVM Classifier with the feature patterns.Step 5: Include non logo frames features in SVM training set.Step 5: The pair of logos can be detected as described in flowchart of Fig. 4.Step 6: Replay is Detected between a pair of logos.Flow Diagram of Testing Phase to Identify an Logo Image in a Sports Video and ReplayDetection Phase is shown in Fig. 44. EXPERIMENTAL RESULTS AND DISCUSSIONSIn the proposed work experimentation is conducted on data set of ten videos compris-ing of soccer and IPL videos. The dataset of all videos logo frames are extracted from all tenvideos and are shown in next page in Fig.5, the features of which are used to train supportvector machine. The SVM trained with these features is employed for detection of pairs oflogos at the beginning and end of each replay action in soccer and cricket videos. The sum-mary of experimentation is presented in Table 1..
  9. 9. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME28Fig. 5 Logo frames trained to SVM classifierThe Table 1 summarizes the performance of proposed algorithm for replay detectionover ten sample sports video out of which 3 are soccer video and 7 are cricket video. The proposedmechanism for replay detects the two logos that sandwich the replay action. The logos are detectedby SVM classifier by testing every frame of the video. The video segment between two consecu-tive logos is picked as replay. The average values of performance parameters recall and precisionup to 94% and 97% for the algorithm indicates that the proposed mechanism is suitable for ex-traction of replay actions in sports video and construct summary of sports videos.
  10. 10. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME29TABLE1 Results of proposed algorithm for 10 test video samples5. CONCLUSIONThe proposed algorithm for SVM based method for replay detection and extractionfrom sports video is designed and experimented with sufficient number of soccer and cricketclips. The algorithm is implemented in Matlab 2010b and executed on Pentium® P6200 pro-cessor with 2GB RAM memory. The replay detection and extraction in soccer and cricketgames are tested well with a sufficient number of video clips. The Proposed algorithms iden-tify a replay by finding a pair of logos and hence extract the replay by extracting all framesin-between the pair of logos. The proposed method identify a pair of logos by matching aentropy features of trained logos with the each frames histogram values of input videos. Theproposed work uses support vector machine to classify between true logo and non logos. Thereplays are used as highlights of a game, and by which we formed a summary of sports videoby these highlights.REFERENCES[1] H. Pan, P. van Beek, and M.I. Sezan, “Detection of slowmotion replay segments in sportsvideo for highlights generation,”in Proc. of ICASSP01, vol. 3, pp. 1649–1652, 2001.[2] X. F. Tong, H. Q. Lu, and Q. S. Liu, “A three-layer event detection framework and itsapplication in soccer video,” in Proc.of ICME04, vol. 3, pp. 1551–1554, 2004.[3] A. Ekin, A. M. Tekalp, and R. Mehrotra, “Automatic soccer video analysis and summari-zation,” IEEE Transactions on Image Processing, vol. 12, no. 7, pp. 796–807, 2003.[4] V. Kobla, D. DeMenthon, and D. Doermann, “Detection of slow-motion replays for iden-tifying sports videos,” in Proc. of MMSP99, pp.135–140, 1999.[5] L. Wang, X. Liu, S. Lin, G. Y. Xu, and H.-Y. Shum, “Generic slow-motion replay detec-tion in sports video,” in Proc. of ICIP04,vol.3, pp. 1585–1588, 2004.[6] J. J. Wang, E. Chng, and C. S. Xu, “Soccer replay detection using scene transition struc-ture analysis,” in Proc. of ICASSP05,vol.2, pp. 433–436, 2005.[7] Y. Yang, S. X. Lin, Y. D. Zhang, and S. Tang, “A statistical framework for replay detec-tion in soccer video,” in Proc. Of ISCAS08, pp. 3538–3541, 2008.Video FileNameActual Num-ber of ReplaysTotal Num-ber of replaycorrectlydetectedreplaysUndetectedreplaysFalsedetectedreplaysRecall PrecisionVideo1.avi 10 9 1 1 90.00% 90.00%Video2.avi 60 56 4 2 93.33% 96.55%Video3.avi 20 20 0 0 100.00% 100.00%Video4.avi 34 31 3 1 91.17% 96.85%Video5.avi 8 7 1 0 87.50% 100.00%Video6.avi 15 14 1 1 93.33% 93.33%Video7.avi 19 19 0 0 100.00% 100.00%Video8.avi 10 10 0 1 100.00% 90.90%Video9.avi 25 23 2 0 92.00% 100.00%Video10.avi 12 12 0 0 100.00% 100.00%
  11. 11. International Journal of Graphics and Multimedia (IJGM), ISSN 0976 – 6448(Print),ISSN 0976 – 6456(Online) Volume 4, Issue 1, January - April 2013, © IAEME30[8] N. Babaguchi, Y. Kawai, Y. Yasugi, and T. Kitahashi, “Linking live and replay scenes inbroadcasted sports video,” in Proc. of MULTIMEDIA00, pp. 205–208, 2000.[9] H. Pan, B. Li, and M. I. Sezan, “Automatic detection of replay segments in broadcastsports programs by detection of logos in scene transitions,” in Proc. of ICASSP02, pp. 3385–3388, 2002.[10] L.-Y. Duan, M. Xu, Q. Tian, and C.-S. Xu, “Mean shift based video segment representa-tion and applications to replay detection,” in Proc. of ICASSP04, pp. 709–712, 2004.[11] X. F. Tong, H. Q. Lu, Q. S. Liu, and H. L. Jin, “Replay detection in broadcasting sportsvideo,” in Proc. of ICIG04, pp.337–340, 2004.[12] Q. Huang, J. M. Hu, W. Hu, T. Wang, H. L. Bai, and Y. M. Zhang, “A reliable logo andreplay detector for sports video,” in Proc. of ICME07, pp. 1695–1698, 2007.[13] Z. H. Dang, J. Du, Q. M. Huang, and S. Q. Jiang, “Replay detection based on semi-automatic logo template sequence extraction in sports video,” in Proc. of ICIG07, pp. 839–844, 2007.[14] B. Han, Y. Yan, Z. H. Chen, C. Liu, and W. G. Wu, “A general framework for automaticon-line replay detection in sports video,” In Proc. of MM09, pp. 501–504, 2009.[15] Reeja S R and Dr. N. P Kavya, “Motion Detection for Video Denoising – The State ofArt and the Challenges”, International Journal of Computer Engineering & Technology(IJCET), Volume 3, Issue 2, 2012, pp. 518 - 525, ISSN Print: 0976 – 6367, ISSN Online:0976 – 6375.