PLC AND SCADA
JAVA AND DOTNET
If ur intrested in these project please feel free to contact us@09640648777,Mallikarjun.V
CHENG et al.: VEHICLE DETECTION IN AERIAL SURVEILLANCE USING DYNAMIC BAYESIAN NETWORKS 2153 Fig. 2. Color histogram of a frame. A. Background Color Removal Since nonvehicle regions cover most parts of the entire scene in aerial images, we construct the color histogram of each frame and remove the colors that appear most frequently in the scene. Fig. 1. Proposed system framework. Take Fig. 2 for example, the colors are quantized into 48 his- togram bins. Among all histogram bins, the 12th, 21st, and 6th bins are the highest and are thus regarded as background colorscolors. The high computational complexity of mean-shift seg- and removed. These removed pixels do not need to be consid-mentation algorithm is another concern. ered in subsequent detection processes. Performing background In this paper, we design a new vehicle detection framework color removal cannot only reduce false alarms but also speed upthat preserves the advantages of the existing works and avoids the detection process.their drawbacks. The modules of the proposed system frame- B. Feature Extractionwork are illustrated in Fig. 1. The framework can be dividedinto the training phase and the detection phase. In the training Feature extraction is performed in both the training phase andphase, we extract multiple features including local edge and the detection phase. We consider local features and color fea-corner features, as well as vehicle colors to train a dynamic tures in this paper.Bayesian network (DBN). In the detection phase, we ﬁrst per- 1) Local Feature Analysis: Corners and edges are usually lo-form background color removal similar to the process proposed cated in pixels with more information. We use the Harris cornerin . Afterward, the same feature extraction procedure is detector  to detect corners. To detect edges, we apply mo-performed as in the training phase. The extracted features serve ment-preserving thresholding method on the classical Cannyas the evidence to infer the unknown state of the trained DBN, edge detector  to select thresholds adaptively according towhich indicates whether a pixel belongs to a vehicle or not. different scenes. In the Canny edge detector, there are two im-In this paper, we do not perform region-based classiﬁcation, portant thresholds, i.e., the lower threshold and the higherwhich would highly depend on results of color segmentation threshold . As the illumination in every aerial image dif-algorithms such as mean shift. There is no need to generate fers, the desired thresholds vary and adaptive thresholds are re-multiscale sliding windows either. The distinguishing feature quired. The computation of Tsai’s moment-preserving methodof the proposed framework is that the detection task is based on  is deterministic without iterations for -level thresholdingpixelwise classiﬁcation. However, the features are extracted in with . Its derivation of thresholds is described as follows.a neighborhood region of each pixel. Therefore, the extracted Let be an image with pixels and denote the grayfeatures comprise not only pixel-level information but also value at pixel . The th moment of is deﬁned asrelationship among neighboring pixels in a region. Such designis more effective and efﬁcient than region-based   ormultiscale sliding window detection methods . The restof this paper is organized as follows: Section II explains the (1)proposed automatic vehicle detection mechanism in detail. where is the total number of pixels in image with graySection III demonstrates and analyzes the experimental results. value and . For bilevel thresholding, we wouldFinally, conclusions are made in Section IV. like to select threshold such that the ﬁrst three moments of image are preserved in the resulting bilevel image . Let all II. PROPOSED VEHICLE DETECTION FRAMEWORK the below-threshold gray values in be replaced by and all Here, we elaborate each module of the proposed system the above-threshold gray values be replaced by ; we can solveframework in detail. for and based on the moment-preserving principle .
2154 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 4, APRIL 2012After obtaining and , the desired threshold is computedusing (2) In order to detect edges, we use the gradient magnitude of each pixel to replace the grayscale value inTsai’s method. Then, the adaptive threshold found by (2) is usedas the higher threshold in the Canny edge detector. We setthe lower threshold as ,where and represent the maximum and minimumgradient magnitudes in the image. Thresholds automatically anddynamically selected by our method give better performanceon the edge detection. We will demonstrate the performanceimprovement on the edge detection with adaptive thresholdsand the corresponding impact on ﬁnal vehicle detection resultsin Section III. 2) Color Transform and Color Classiﬁcation: In , theauthors proposed a new color model to separate vehicle colorsfrom nonvehicle colors effectively. This color model transforms color components into the color domain , i.e., Fig. 3. Vehicle colors and nonvehicle colors in different color spaces. (a) , (3) (b) , (c) , and (d) planes. (4)where is the , , and color components ofpixel and . It has been shownin  that all the vehicle colors are concentrated in a muchsmaller area on the plane than in other color spaces andare therefore easier to be separated from nonvehicle colors.Although the color transform proposed in  did not aim foraerial images, we have found that the separability propertystill presents in aerial images. As shown in Fig. 3, we canobserve that vehicle colors and nonvehicle colors have lessoverlapping regions under the color model. Therefore,we apply the color transform to obtain componentsﬁrst and then use a support vector machine (SVM) to classifyvehicle colors and nonvehicle colors. When performing SVM Fig. 4. Neighborhood region for feature extraction.training and classiﬁcation, we take a block of pixelsas a sample. More speciﬁcally, each feature vector is deﬁnedas . Notice that we do not perform Features and are deﬁned, respectively, asvehicle color classiﬁcation via SVM for blocks that do not con-tain any local features. Those blocks are taken as nonvehicle (6)color areas. As we mentioned in Section I, the features are extracted in (7)a neighborhood region of each pixel in our framework. Consid-ering an neighborhood of pixel , as shown in Fig. 4, Similarly, denotes to the number of pixels in thatwe extract ﬁve types of features, i.e., , , , , and , for are detected as corners by the Harris corner detector, andthe pixel. These features serve as the observations to infer the denotes the number of pixels in that are detected as edges byunknown state of a DBN, which will be elaborated in the next the enhanced Canny edge detector. The pixels that are classiﬁedsubsection. The ﬁrst feature denotes the percentage of pixels as vehicle colors are labeled as connected vehicle-color regions.in that are classiﬁed as vehicle colors by SVM, as deﬁned in The last two features and are deﬁned as the aspect ratio(5). Note that denotes to the number of pixels in and the size of the connected vehicle-color region where the that are classiﬁed as vehicle colors by SVM, i.e., pixel resides, as illustrated in Fig. 4. More speciﬁcally, , and feature is the pixel count of “vehicle- (5) color region 1” in Fig. 4.
CHENG et al.: VEHICLE DETECTION IN AERIAL SURVEILLANCE USING DYNAMIC BAYESIAN NETWORKS 2155 Fig. 5. DBN model for pixelwise classiﬁcation.C. DBN We perform pixelwise classiﬁcation for vehicle detectionusing DBNs . The design of the DBN model is illustratedin Fig. 5. Node indicates if a pixel belongs to a vehicle attime slice . The state of is dependent on the state of .Moreover, at each time slice , state has inﬂuences on theobservation nodes , , , , and . The observationsare assumed to be independent of one another. The deﬁnitionsof these observations are explained in the previous subsection.Discrete observation symbols are used in our system. We use -means to cluster each observation into three clusters, i.e.,we use three discrete symbols for each observation node. In thetraining stage, we obtain the conditional probability tables ofthe DBN model via expectation-maximization  algorithmby providing the ground-truth labeling of each pixel and itscorresponding observed features from several training videos.In the detection phase, the Bayesian rule is used to obtain theprobability that a pixel belongs to a vehicle, i.e., (8) Fig. 6. Snapshots of the experimental videos. The joint probability is theprobability that a pixel belongs to a vehicle pixel at time slice given all the observations and the state of the previous timeinstance. According to the naive Bayesian rule of conditionalprobability, the desired joint probability can be factorizedsince all the observations are assumed to be independent. Term is deﬁned as the probability that a pixel belongs toa vehicle pixel at time slice given observation at timeinstance [ is deﬁned in (5)]. Terms , , Fig. 7. Background color removal results. , , and are similarly deﬁned. The proposed vehicle detection framework can also utilize aBayesian network (BN) to classify a pixel as a vehicle or non- III. EXPERIMENTAL RESULTSvehicle pixel. When performing vehicle detection using BN, thestructure of the BN is set as one time slice of the DBN model Experimental results are demonstrated here. To analyze thein Fig. 5. We will compare the detection results using BN and performance of the proposed system, various video sequencesDBN in the next section. with different scenes and different ﬁlming altitudes are used. The experimental videos are displayed in Fig. 6. Note that itD. Post Processing is infeasible to assume prior information of camera heights and target object sizes for this challenging data set. When We use morphological operations to enhance the detection performing background color removal, we quantize the colormask and perform connected component labeling to get the ve- histogram bins as 16 16 16. Colors corresponding to thehicle objects. The size and the aspect ratio constraints are ap- ﬁrst eight highest bins are regarded as background colors andplied again after morphological operations in the postprocessing removed from the scene. Fig. 7(a) displays an original imagestage to eliminate objects that are impossible to be vehicles. frame, and Fig. 7(b) displays the corresponding image afterHowever, the constraints used here are very loose. background removal.
2156 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 4, APRIL 2012 TABLE I DETECTION ACCURACY USING DIFFERENT NEIGHBORHOOD SIZESFig. 8. Training images for vehicle color classiﬁcation: (a) Positive samplesand (b) negative samples. Fig. 9. Inﬂuence of different block sizes on the detection rate. Fig. 11. Impact of the enhanced edge detector on vehicle detection results. whose ground-truth vehicle positions are manually marked. That is, we use only 180 frames to train the DBN. To select the size of the neighborhood area for feature extraction, we list the detection accuracy using different neighborhood sizes in Table I. The detection accuracy is measured by the hit rate and the number of false positives per frame. There are a total of 225 025 frames in the data set. When calculating the detection accuracy, we perform evaluation every 100 frames. We can observe that the neighborhood area with the size of 7 7 Fig. 10. Results of vehicle color classiﬁcation. yields the best detection accuracy. Therefore, for the rest of the experiments, the size of the neighborhood area for extracting observations is set as 7 7. When employing SVM, we need to select the block size The impacts of the enhanced Canny edge detector on vehicleto form a sample (see Fig. 8)<<AQ1>>. Fig. 9 displays the inﬂu- detection results can be observed in Fig. 11. Fig. 11(a) shows theence of different block sizes on the detection rate. According to results obtained using the traditional Canny edge detector withthe observation from Fig. 9, we take each 3 4 block to form a nonadaptive thresholds. Fig. 11(b) shows the detection resultsfeature vector. The color of each pixel would be transformed to obtained using the enhanced Canny edge detector with moment- and color components using (3) and (4). More speciﬁcally, preserving threshold selection. Nonadaptive thresholds cannoteach feature vector has a dimension of 24. The training images adjust to different scenes and would therefore result in moreused to train the SVM are displayed in Fig. 8. Notice that the misses or false alarms, as displayed in Fig. 11(a). To examine theblocks that do not contain any local features are taken as non- necessity of the background removal process and the enhancedvehicle areas without the need of performing classiﬁcation via edge detector, we list the detection accuracy of four differentSVM. Fig. 10 shows the results of color classiﬁcation by SVM scenarios in Table II. We can observe that the background re-after background color removal and local feature analysis. moval process is important for reducing false positives and the To obtain the conditional probability tables of the DBN, we enhanced edge detector is essential for increasing hit rates.select the training clips from the ﬁrst six experimental videos We compare different vehicle detection methods in Fig. 12.displayed in Fig. 6. The remaining four videos are not involved The moving-vehicle detection with road detection method in in the training process. Each training clips contains 30 frames, requires setting a lot of parameters to enforce the size constraints
CHENG et al.: VEHICLE DETECTION IN AERIAL SURVEILLANCE USING DYNAMIC BAYESIAN NETWORKS 2157 TABLE II DETECTION ACCURACY OF FOUR DIFFERENT SCENARIOS Fig. 12. Comparisons of different vehicle detection methods.in order to reduce false alarms. However, for the experimentaldata set, it is very difﬁcult to select one set of parameters that Fig. 13. Vehicle detection results: (a) BNs and (b) DBNs.suits all videos. Setting the parameters heuristically for the dataset would result in low hit rate and high false positive num-bers. The cascade classiﬁers used in  need to be trained by demonstrate ﬂexibility and good generalization ability on a widea large number of positive and negative training samples. The variety of aerial surveillance scenes under different heights andnumber of training samples required in  is much larger than camera angles. It can be expected that the performance of DBNthe training samples used to train the SVM classiﬁer in Fig. 8. is better than that of the BN. In Fig. 13, we display the detectionThe colors of the vehicles would not dramatically change due to results using BN and DBN [see Fig. 13(a) and (b), respectively].the inﬂuence of the camera angles and heights. However, the en- The colored pixels are the ones that are classiﬁed as vehicletire appearance of the vehicle templates would vary a lot under pixels by BN or DBN. The ellipses are the ﬁnal vehicle detec-different heights and camera angles. When training the cascade tion results after performing postprocessing. DBN outperformsclassiﬁers, the large variance in the appearance of the positive BN because it includes information along time. When observingtemplates would decrease the hit rate and increase the number detection results of consecutive frames, we also notice that theof false positives. Moreover, if the aspect ratio of the multiscale detection results via DBN are more stable. The reason is that,detection windows is ﬁxed, large and rotated vehicles would be in aerial surveillance, the aircraft carrying the camera usuallyoften missed. The symmetric property method proposed in  follows the vehicles on the ground, and therefore, the positionsis prone to false detections such as symmetrical details of build- of the vehicles would not have dramatic changes in the sceneings or road markings. Moreover, the shape descriptor used to even when the vehicles are moving in high speeds. Therefore,verify the shape of the candidates is obtained from a ﬁxed ve- the information along the time contributed by helpshicle model and is therefore not ﬂexible. Moreover, in some stabilize the detection results in the DBN.of our experimental data, the vehicles are not completely sym- Fig. 14 shows some detection error cases. Fig. 14(a) displaymetric due to the angle of the camera. Therefore, the method in the original image frames, and Fig. 14(b) displays the detec- is not able to yield satisfactory results. tion results. The black arrows in Fig. 14(b) indicate the miss Compared with these methods, the proposed vehicle detec- detection or false positive cases. In the ﬁrst row of Fig. 14, thetion framework does not depend on strict vehicle size or as- rectangular structures on the buildings are very similar to vehi-pect ratio constraints. Instead, these constraints are observa- cles. Therefore, sometimes, these rectangular structures wouldtions that can be learned by BN or DBN. The training process be detected as vehicles incorrectly. In the second row of Fig. 14,does not require a large amount of training samples. The results the miss detection is caused by the low contrast and the small
2158 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 21, NO. 4, APRIL 2012 detecting rotated vehicles either. Instead, we have proposed a pixelwise classiﬁcation method for the vehicle detection using DBNs. In spite of performing pixelwise classiﬁcation, relations among neighboring pixels in a region are preserved in the fea- ture extraction process. Therefore, the extracted features com- prise not only pixel-level information but also region-level in- formation. Since the colors of the vehicles would not dramat- ically change due to the inﬂuence of the camera angles and heights, we use only a small number of positive and negative samples to train the SVM for vehicle color classiﬁcation. More- over, the number of frames required to train the DBN is very small. Overall, the entire framework does not require a large amount of training samples. We have also applied moment pre- serving to enhance the Canny edge detector, which increases the adaptability and the accuracy for detection in various aerial im- Fig. 14. Detection error cases. ages. The experimental results demonstrate ﬂexibility and good generalization abilities of the proposed method on a challenging data set with aerial surveillance images taken at different heights and under different camera angles. For future work, performing vehicle tracking on the detected vehicles can further stabilize the detection results. Automatic vehicle detection and tracking could serve as the foundation for event analysis in intelligent aerial surveillance systems. REFERENCES  R. Kumar, H. Sawhney, S. Samarasekera, S. Hsu, T. Hai, G. Yanlin, K. Hanna, A. Pope, R. Wildes, D. Hirvonen, M. Hansen, and P. Burt, “Aerial video surveillance and exploitation,” Proc. IEEE, vol. 89, no. 10, pp. 1518–1539, 2001.  I. Emst, S. Sujew, K. U. Thiessenhusen, M. Hetscher, S. Rabmann, and M. Ruhe, “LUMOS—Airbome trafﬁc monitoring system,” in Proc. IEEE Intell. Transp. Syst., Oct. 2003, vol. 1, pp. 753–759.  L. D. Chou, J. Y. Yang, Y. C. Hsieh, D. C. Chang, and C. F. Tung, “In- Fig. 15. Processing speeds of different methods. tersection-based routing protocol for VANETs,” Wirel. Pers. Commun., vol. 60, no. 1, pp. 105–124, Sep. 2011.  S. Srinivasan, H. Latchman, J. Shea, T. Wong, and J. McNair, “Air- borne trafﬁc surveillance systems: Video surveillance of highwaysize of the vehicle. However, other vehicles are successfully de- trafﬁc,” in Proc. ACM 2nd Int. Workshop Video Surveillance Sens.tected in this challenging setting. Netw., 2004, pp. 131–135. Fig. 15 shows the average processing speeds of different ve-  A. C. Shastry and R. A. Schowengerdt, “Airborne video registration and trafﬁc-ﬂow parameter estimation,” IEEE Trans. Intell. Transp.hicle detection methods. The experiments are conducted on a Syst., vol. 6, no. 4, pp. 391–405, Dec. 2005.personal computer with 2.66-GB central processing unit and  H. Cheng and J. Wus, “Adaptive region of interest estimation for aerial4-G random access memory. The frame size of the experimental surveillance video,” in Proc. IEEE Int. Conf. Image Process., 2005, vol. 3, pp. 860–863.videos is 640 480 pixels. Although the proposed framework  S. Hinz and A. Baumgartner, “Vehicle detection in aerial images usingusing BN and DBN cannot reach the frame rate of the surveil- generic features, grouping, and context,” in Proc. DAGM-Symp., Sep.lance videos, it is sufﬁcient to perform vehicle detection every 2001, vol. 2191, Lecture Notes in Computer Science, pp. 45–52.  H. Cheng and D. Butler, “Segmentation of aerial surveillance video50–100 frames. Tracking algorithms can be applied on the in- using a mixture of experts,” in Proc. IEEE Digit. Imaging Comput.termediate frames between two detection frames to track each —Tech. Appl., 2005, p. 66.individual vehicle. Therefore, high detection rate and low false  R. Lin, X. Cao, Y. Xu, C. Wu, and H. Qiao, “Airborne moving vehicle detection for urban trafﬁc surveillance,” in Proc. 11th Int. IEEE Conf.alarm rate should be the primary considerations of designing de- Intell. Transp. Syst., Oct. 2008, pp. 163–167.tection methods given the condition that the execution time is  L. Hong, Y. Ruan, W. Li, D. Wicker, and J. Layne, “Energy-basedreasonable. video tracking using joint target density processing with an application to unmanned aerial vehicle surveillance,” IET Comput. Vis., vol. 2, no. 1, pp. 1–12, 2008. IV. CONCLUSION  R. Lin, X. Cao, Y. Xu, C. Wu, and H. Qiao, “Airborne moving vehicle detection for video surveillance of urban trafﬁc,” in Proc. IEEE Intell. In this paper, we have proposed an automatic vehicle detec- Veh. Symp., 2009, pp. 203–208.tion system for aerial surveillance that does not assume any prior  J. Y. Choi and Y. K. Yang, “Vehicle detection from aerial images using local shape information,” Adv. Image Video Technol., vol. 5414, Lec-information of camera heights, vehicle sizes, and aspect ratios. ture Notes in Computer Science, pp. 227–236, Jan. 2009.In this system, we have not performed region-based classiﬁca-  C. G. Harris and M. J. Stephens, “A combined corner and edge de-tion, which would highly depend on computational intensive tector,” in Proc. 4th Alvey Vis. Conf., 1988, pp. 147–151.  J. F. Canny, “A computational approach to edge detection,” IEEEcolor segmentation algorithms such as mean shift. We have not Trans. Pattern Anal. Mach. Intell., vol. PAMI-8, no. 6, pp. 679–698,generated multiscale sliding windows that are not suitable for Nov. 1986.
CHENG et al.: VEHICLE DETECTION IN AERIAL SURVEILLANCE USING DYNAMIC BAYESIAN NETWORKS 2159  W. H. Tsai, “Moment-preserving thresholding: A new approach,” Chih-Chia Weng was born in Taiwan. He received Comput. Vis. Graph., Image Process., vol. 29, no. 3, pp. 377–393, 1985. the B.S. degree in computer science and information  L. W. Tsai, J. W. Hsieh, and K. C. Fan, “Vehicle detection using nor- engineering and the M.S. degree from Chung Cheng malized color and edge map,” IEEE Trans. Image Process., vol. 16, no. Institute of Technology of National Defense Univer- 3, pp. 850–864, Mar. 2007. sity, Taoyuan, Taiwan. He is currently working to-  N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector ward the Ph.D. degree in the Department of Com- Machines and Other Kernel-Based Learning Methods. Cambridge, puter Science and Information Engineering, National U.K.: Cambridge Univ. Press, 2000. Central University, Chungli, Taiwan.  S. Russell and P. Norvig, Artiﬁcial Intelligence: A Modern Approach, His research interest includes image processing 2nd ed. Englewood Cliffs, NJ: Prentice-Hall, 2003. and machine intelligence. Hsu-Yung Cheng (M’08) was born in Taipei, Taiwan. She received the B.S. degree in computer science and information engineering and M.S. Yi-Ying Chen was born in Taiwan. She received the degree from the National Chiao Tung University, B.S. degree in computer science from the National Hsinchu, Taiwan, in 2000 and 2002, respectively, Taipei University of Education, Taipei, Taiwan, and and the Ph.D. degree in electrical engineering from the M.S. degree from the National Central University, the University of Washington, Seattle, in 2008. Chungli, Taiwan. Since 2008, he has been an Assistant Professor Her research interest includes signal and image with the Department of Computer Science and In- processing. formation Engineering, National Central University, Chung-li, Taiwan. Her research interest includesimage and video analysis and intelligent systems.