Video based distance traffic analysis application to vehicle tracking and counting


Published on

services on......
If ur intrested in these project please feel free to contact us@09640648777,Mallikarjun.V

Published in: Technology
1 Comment
No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Video based distance traffic analysis application to vehicle tracking and counting

  1. 1. Scientific Image ProcessingVideo-Based Distance TrafficAnalysis: Application to VehicleTracking and CountingImaging hardware and video processing techniques offer advantages for traffic monitoringand surveillance. As these experiments show, using appropriate road and vehicle modelingand strong vehicle detection and tracking algorithms offers a good trade-off betweencorrect dynamic vehicle identification and a real-time frame rate. Further, heuristic rules canhelp analyze and solve difficult traffic situations. G iven the increasing social de- Depending on the camera position,2 there are mands of mobility and safety in two common scenarios for video-based traffic road transportation, many gov- analysis: ernments are making the devel- opment of automatic, economic, and real-time • one or more static cameras are placed at a cer- solutions for reliable traffic flow analysis a pri- tain height and distance to offer a good per- ority. As video cameras become relatively inex- spective of the controlled region, or pensive, applying video processing techniques • onboard cameras within vehicles are used for to traffic monitoring and analysis is now driver-warning applications. cheaper and less disruptive than installing loop detectors.1 Our work corresponds to the first case and is A key goal of automatic, video-based traffic based on three system requirements. It must analysis is to detect and track vehicles driving through a controlled area and thus identify • be low cost, using only one static camera placed abnormal events, such as traffic congestion, on one side of the road, at some distance; speeding violations and other illegal driving • offer a good trade-off between vehicle tracking behaviors, and even accidents. Using video, accuracy and computational efficiency; and it’s possible to compute traffic measures of • provide accurate vehicle counting, even in com- involved vehicles, including their speeds, plex traffic scenes (such as on busy roads or with types, or overall numbers in the analyzed road partial vehicle occlusions). region.1 These requirements complicate the problem1521-9615/11/$26.00 © 2011 IEEE because tracking vehicles over time relies on aCopublished by the IEEE CS and the AIP good segmentation of them. Moreover, the cam-Angel Sánchez and Pedro D. Suárez era’s distance from the scene and the perspec-Rey Juan Carlos University, Madrid tive effect further increase the challenges. Here, we describe how our proposed system addressesAura Conci and Eldman O. Nunes these challenges and discuss our highly promisingFluminense Federal University, Rio de Janeiro experimental results.38 This article has been peer-reviewed. Computing in Science Engineering
  2. 2. System OverviewKey components of our system are the roadmodel and the vehicle model for detecting andtracking vehicles along the video sequence, andthe proposed algorithm for automatic trafficanalysis.Road ModelOur road model includes components that makeknowledge of the problem domain explicit, in-cluding the enter and exit regions, rail regions,road regions, image perspective, and obstacles.Enter and exit regions. The enter and exit areasare predefined (and fixed) image regions wherethe vehicles enter and abandon the analyzedscene, respectively. The regions are rectangu- Figure 1. Multilane road model with imagelar, narrow strip areas that are perpendicular geometry. The system uses one static, uncalibrated camera placed at a certain height on one sideto the road verges. After computing the binary of the road (but not vertical to it). The road isimage resulting from a background subtraction approximately oriented along the camera’s depthbetween the actual frame and the background axis.image, the system checks if the number of in-tersecting pixels between the area of an enter In the exiting condition, an identified vehicle vor exit region EE(i) with a connected compo- exits the scene across the exit region EE(i) whennent ConnectedComponent(v) exceeds a pre- the following condition holds:defined region threshold thEE(i). In this case (seeEquation 1), the corresponding enter or exit re- Exiting(EE(i),v) = Activation(EE(i),v)gion EE(i) (i = 1, 2, .., K) is activated at time t and AND ActiveTarget(v) (3)the connected component v becomes a detectedactive target—that is, a vehicle not previously Figure 1 shows a road example with one enterdetected— or ActiveTarget(v): region (green line) and one exit region (blue line).Activation(EE(i),v) = Area(EE(i)) ∩ ConnectedComponent(v) ≥ thEE(i) (1) Rail regions. One rail region is defined for each road lane. Rail regions help locate and track, along In our model, enter regions can’t work simul- time, the set of vehicles present at the controlledtaneously as exit regions (and vice versa). More road region. Rail regions are approximatelythan one of these regions can be activated at the straight lines from the camera image’s perspective.same time t. If an active target (that is, a detected As we describe later, rails are also used to solve sit-vehicle with an assigned identification number) uations that are difficult to analyze. For example,that was tracked a long time reaches an exit area, several cars might share the same connected com-we interpret that this target is exiting the scene. ponent (the same white blob) on a frame, and theOn the other hand, if no active target exists yet, region corresponding to each contained car mustand a number of pixels exceeding the predefined be delimited. The activation condition of a rail re-threshold thEE(i) appears at an enter area EE(i), a gion at frame t when a vehicle crosses it becomesnew blob is produced by a detected vehicle, and similar to the expression of Equation 1. Figure 1its state is included on the list of detected vehicles. also shows in the same road example four rail re-The entering and exiting conditions are as follows gions (white lines).(see Equations 2 and 3). In the entering condition,a connected component v enters the scene across Road region. The road region represents an im-the enter region EE(i); it’s considered a new ve- portant part of the problem domain knowledgehicle when and can thus be useful in detecting traffic events. It consists of the set of road pixels from the cameraEntering(EE(i),v) = Activation(EE(i),v) image’s perspective and contains enter, exit, and AND NOT ActiveTarget(v) (2) rail regions.May/June 2011 39
  3. 3. Related Work: Using and thus makes correct detection and counting more difficult. Video in Vehicle Detection and Tracking References 1. R. Cucchiara, M. Piccardi, and P. Mello, “Image Analysis and T he literature contains several related works on automatic video analysis for vehicle detection and tracking.1,2 Many researchers use some type of background subtrac- Rule-Based Reasoning for Traffic Monitoring System,” IEEE Trans. Intelligent Transportation Systems, vol. 1, no. 2, 2000, pp. 119–130. 2. V. Kastrinaki, M. Zervakis, and K. Kalaitzakis, “A Survey of Video tion method to segment vehicles in a scene.3 With respect Processing Techniques for Traffic Applications,” Image and Vision to vehicle tracking, some works apply different optical Computing, vol. 21, no. 4, 2003, pp. 359–381. flow methods.2,4 Other researchers apply predictive filter- 3. S.C. Cheung and C. Kamath, “Robust Techniques for Background ing techniques like Kalman5 or particle filters6 to track the Subtraction in Urban Traffic Video,” Proc. Int’l Conf. Visual detected vehicles. However, these approaches—which are Communications and Image Processing, Int’l Soc. for Optonics and based on sophisticated probabilistic model estimation Photonics (SPIE), 2004, pp. 881–892. techniques—usually decrease their performance and accu- 4. B. Li and R. Chellappa, “A Generic Approach to Simultaneous racy when forced to simultaneously track a high number of Tracking and Verification in Video,” IEEE Trans. Image Processing, targets. vol. 11, no. 5, 2002, pp. 530–544. Other works consider the problem of handling occlu- 5. J.W. Hsieh et al., “Automatic Traffic Surveillance Systems for sions in the scenes (because of perspective, road signs, Vehicle Tracking and Classification,” Proc. IEEE Conf. Intelligent or the presence of other vehicles). In particular, Camillo Transportation Systems, vol. 7, 2006, pp. 175–187. Gentile, Octavia I. Camps, and Mario Sznaier proposed 6. B. Ristic, S. Arulampalam, and N. Gordon, Beyond the Kalman tracking vehicles as a set of parts to solve some types Filter: Particle Filters for Tracking Applications, Artech House, 2004. of partial occlusions.7 Other researchers reported using 7. C. Gentile, O. Camps, and M. Sznaier, “Segmentation for Robust multiple cameras for 3D detection and tracking multiple Tracking in the Presence of Severe Occlusion,” IEEE Trans. Image vehicles.8 Although these systems can interpret more Processing, vol. 13, no. 2, 2004, pp. 166–178. complex traffic scenes, they’re also much more expen- 8. Q. Houben et al., “Multi-Feature Stereo Vision System for Road sive and require calibration algorithms. Finally, Guohui Traffic Analysis,” Proc. Int’l Conf. Computer Vision Theory and Zhang, Ryan P. Avery, and Yinhai Wang also used one Applications (VISAPP), vol. 2, Inst. Systems and Technologies of uncalibrated surveillance video camera in their work.9 Information, Control, and Comm. (INSTICC) Press, 2009, Their approach is also cost effective, uses an adaptive pp. 554–559. background approach, and counts in real time the ve- 9. G. Zhang, R.P. Avery, and Y. Wang, “A Video-Based Vehicle hicles in each lane of the roadway. However, in their sys- Detection and Classification System for Real-Time Traffic tem, the camera is placed on a bridge directly above the Data Collection Using Uncalibrated Cameras,” Transportation road such that the video-sequences aren’t affected by Research Record: J. Transportation Research Board, vol. 1993, 2007, the perspective that produces more vehicle occlusions, pp. 138–147, doi:10.3141/1993-19. Image perspective. As Figure 1 shows, the system the visibility of some tracked vehicles during sev- configuration uses one static, uncalibrated camera eral frames. However, as we describe later, the placed at a certain height and at one side of the connected regions associated with the vehicles road (but not vertical to it). The road is approxi- can be recovered again by taking into account a mately oriented along the camera’s depth axis; set of heuristic rules. the vehicles are either farther from or closer to the camera and thus should be seen as smaller or Vehicle Model: Detection and Tracking larger in size. Consequently, each vehicle blob size In our model, each vehicle moving along the ana- and shape depends on the camera’s viewpoint with lyzed road region is characterized by a unique respect to the road, which makes it more difficult identification number that’s assigned after it’s first to correctly detect vehicles than when the camera detected as passing by any enter region. (To learn is vertically placed just over the road.3 about other research in this area, see the sidebar detailing related work.) The state of a tracked Obstacles. In our model, obstacles correspond vehicle st at frame t consists of its identification to static elements outside the road region, such number, id, which is a consecutive number as- as billboards and other signs that are projected signed after detecting the vehicle passing through inside the analyzed road region because of an enter region; a Boolean flag ft indicating camera perspective. These obstacles could affect whether the vehicle’s current position is known40 Computing in Science Engineering
  4. 4. at frame t; two spatial coordinates denoting the background application, we considered and testedtarget’s center-of-mass position (xt and yt, respec- various approaches.4 We then chose the approxi-tively); two scalars representing the sides of the mate median algorithm because of its simplicityvehicle bounding box (Lxt and Lyt, respectively); and its good performance in our application.5 Inand the corresponding velocity coordinates (vx,t this method, if a pixel in the current frame has aand vy,t, respectively). Therefore, this vehicle state brightness value larger than the correspondingst at frame t is represented in Equation 4 by the background pixel, the background pixel is incre-eight-tuple mented by one; otherwise, the corresponding background pixel is decremented by one. This isst = (id, ft, xt, yt, Lxt, Lyt, vx,t, vy,t). independently applied for each color channel in (4) the frames of the traffic videos. Consequently, the scene background image will converge to an es- The state of the whole set of the active vehicles timate where approximately half the input pixelsin the road is a variable-length list consisting of will be larger than the background, and half willthe involved vehicles’ states. At each frame, the be smaller than the background (that is, the me-position and bounding box size corresponding to dian value). The convergence time will depend oneach vehicle are properly updated. The approxi- the frame rate and on the amount of movement inmate vehicle position is its main identification the scene.information along the tracking. In addition to po- We initially set the list of detected vehicles tosition, its bounding box size is obtained using a empty. For each frame in the video sequence, abackground subtraction technique. We have useda near-constant velocity motion model in Equa-tion 5 to compute the position and velocity of each For each detected vehicle in the previous frame,moving target for each frame: a white blob in the present frame is tentativelyxt+δt = xt + vx,t δt + Fyt+δt = yt + vy,t δt + F (5)vx,t+δt = vx,t + G assigned to it. If there’s an overlap betweenvy,t+δt = vy,t + G a vehicle’s bounding boxes at the currentwhere dt is the time step defining the frame video and previous frame, the vehicle position israte, and F and G are two excitation forces mod-eled by random uniform variables in a given checked and updated according to the vehiclerange ([minF, maxF ] and [minG, maxG], respec-tively), which permit the reduction and control movement model.of the modifications in the vehicle’s position andvelocity.High-Level Pseudocode for Traffic Analysis background subtraction algorithm is used to de-The following algorithm pseudocode offers a tect the moving objects in the scene. This step ishigh-level description of our vehicle detection and computed by applying the corresponding statictracking system. We consider two approaches for or adaptive background subtraction method; thebackground subtraction: result is a bilevel image where detected moving objects (or blobs) appear as connected regions of• a simple static approach, in which the back- white pixels (see Figure 2b). ground image is fixed along the video sequence; For each detected vehicle in the previous frame and (which is stored in the ListDetected structure),• an adaptive approach, in which the background a white blob in the present frame is tentatively image will adapt to illumination, motion, and assigned to it. If there’s an overlap between a scene geometry changes. vehicle’s bounding boxes at the current and previ- ous frame, the vehicle position is checked and up- In the static background approach, we detect dated according to the vehicle movement model;foreground moving targets by computing the otherwise, the vehicle’s position is considered asdifference between each video frame and the temporally “lost.” Next, as we describe later, wefixed scene background image. For the adaptive solve any abnormal situations produced in theMay/June 2011 41
  5. 5. (a) (b) Figure 2. Analyzing traffic. (a) A sample frame with rail lines drawn (in white). (b) The corresponding binary image obtained by background subtraction containing the detected targets (rail lines also help in a more robust detection of these targets). listDetected:=∅; {list of detected vehicles} background0:= SetInitialBackground(video); for i:=1 to NumFrames do {number of analyzed frames} read(framei); backgroundi=UpdateBackground(framei,backgroundi-1); imageBW:=BackgroundSubtraction(framei,backgroundi); for each vehicle in listDetected do EstimateBoundingBox(framei,vehicle); overlap:=CheckOvrlapBox(imageBW,imagePrevBW,vehicle); if overlap then{analyzed vehicle is found} UpdateVehiclePos(vehicle,imageBW,imagePrevBW) else vehicle=lost {vehicle position is actually lost} end; HandleConflictCases(imageBW,listDetected); {Section 3} UpdateListDetectEnter(listDetected,imageBW); UpdateListDetectExit(listDetected,imageBW); imagePrevBW:=imageBW end; Figure 3. A high-level algorithmic description of the proposed traffic analysis method using the adaptive background subtraction approach. current frame. The system uses the road model Traffic Analysis for Difficult Situations we described earlier to solve the problem of con- Our framework considers several use cases cor- nected components corresponding to several ve- responding to difficult vehicle tracking situa- hicles. Once the information of tracked vehicles tions. Rule-based reasoning is applied on these is updated, the processing of the current frame scenarios. Here, we focus on two possible situ- ends with the analysis of the road’s enter and exit ations in our video-based traffic monitoring regions to check whether new vehicles appear or prototype: a known vehicle leaves the scene, respectively. Figure 3 shows a pseudocode of the vehicle detec- • several vehicles are so close to each other tion and tracking method. in some frames that only one white blob42 Computing in Science Engineering
  6. 6. IF two or more vehicles share the same blob (or connected component) THEN partition the set of vehicles into subsets by their closeness to the rail lines; FOR EACH subset of vehicles close to the same rail line DO consider the position of vehicles at previous frame and determine the positional ordering of the vehicle identifications; assign the same identification number to a vehicle of the current frame than in the previous one (preserving the positional ordering of previous frame); divide the whole connected component into subregions (one for each involved vehicle) by assigning the pixels of the blob to the closest centroids.Figure 4. Pseudocode for Rule 1: Several vehicles in the same connected component. WHILE the connected component of a vehicle gets lost DO predict vehicle position at current frame considering its previous position and velocity vector; perform a local search of connected components not assigned to vehicles around the new vehicle position; IF a non-assigned vehicle component is found THEN assign it to the analyzed vehicle ELSE keep the vehicle position flag ft (Equation 4) as temporally unknown.Figure 5. Pseudocode for Rule 2: A vehicle “loses” its connected component during the tracking time. represents them (after background subtrac- • Rule 2: A vehicle “loses” its connected compo- tion); and nent (bounding box) during the tracking time.• one previously detected vehicle is lost at some • Description: Because of the image perspective, point during the tracking and has no connected external elements (advertising billboards) are component associated to it. projected inside the analyzed road region and can thus partially or completely occlude someThe first use case can occur if one vehicle vehicles in the scene during several frames.shape is projected onto another because of • Precondition: The vehicle’s position is known atimage perspective or to reduced contrast be- the previous frame.tween vehicle colors and the asphalt. Thesecond use case can occur when the projec- Figure 5 shows the pseudocode for Rule 2.tion of a static obstacle in the road regioncauses the car to “disappear” from the camera’s Performance Evaluationviewpoint. To evaluate the proposed traffic analysis system’s ac- We now briefly describe the rules implement- curacy for detecting and tracking vehicles, we useding the adopted solutions for both cases. different live video recordings taken at different time periods. An AXIS Q1755 camera provided• Rule 1: Several vehicles in the same connected images with a spatial resolution of 384 × 356; component (bounding box). we obtained them directly from the AXIS web-• Description: The separated blobs of tracked vehi- site demo gallery ( cles in previous frames appear now as grouped gallery.htm). For our experiments, we used im- in the same connected component at the cur- ages provided by one static camera located over rent frame. Petrovka Street in Kiev, Ukraine.• Precondition: The number of involved vehicles All the algorithms were implemented in C++ and the corresponding position of centroids are on a PC AMD Athlon 64 × 2 Dual Core Pro- known at the current frame. cessor 4000+ with 2.10 GHz and 1 Gbyte RAM under Windows OS. The system prototype runsFigure 4 shows the pseudocode for Rule 1. at real time. Moreover, it can run up to 50 framesMay/June 2011 43
  7. 7. (a) (b) (c) (d) Figure 6. Four sample frames from traffic tracking. These visual results are for a 27-second test video sequence. Table 1. Vehicles detected using the static and adaptive approaches. Static background Adaptive background Sequence ID Inspection Algorithm Detection Algorithm Detection (frames interval) counts counts rate (%) counts rate (%) Sequence 1 (251–500) 20 22 90.0 23 85.0 Sequence 2 (501–750) 16 17 93.7 16 100.0 Sequence 3 (751–1,000) 11 12 90.9 12 90.9 Sequence 4 (1,001–1,250) 13 14 92.3 14 92.3 per second (fps) for the considered image resolu- in red near the boxes (we kept these identifica- tion. Figure 6 shows four images from the detec- tion numbers for the cars throughout the tracking tion results from an example video sequence of process). 1,250 frames. Detected vehicles are enclosed by Table 1 shows the numerical results for four bounding boxes (yellow rectangles), with their disjoint subsequences of the analyzed video. corresponding identification numbers displayed We compared the number of automatically44 Computing in Science Engineering
  8. 8. detected vehicles (using the static and adaptive Applications,” Image and Vision Computing, vol. 21,background detection methods) to the number of no. 4, 2003, pp. 359–381.vehicles detected by human visual count for the 2. E. Bas, M. Tekalp, and F.S. Salman, “Automatic Ve-same sequence. The adaptive background algo- hicle Counting from Video for Traffic Flow Analysis,”rithm required approximately 250 frames for its Proc. IEEE Intelligent Vehicles Symp., IEEE Press, 2007,initialization. After that, both background detec- pp. 392–397.tion approaches produced similar correct vehicle 3. G. Zhang, R.P. Avery, and Y. Wang, “A Video-Baseddetection results for the analyzed subsequences. Vehicle Detection and Classification System for Real-On average, we achieved 91.7 percent success us- Time Traffic Data Collection Using Uncalibrateding the static background and 92.1 percent using Cameras,” J. Transportation Research Board, IEEE Press,the adaptive (approximate median) method. 2007, pp. 138–147. To complement the validation of our approach, 4. S.C. Cheung and C. Kamath, “Robust Techniqueswe also tested it on a second video sequence: a for Background Subtraction in Urban Traffic Video,”roundabout scene at 30 fps in which the frames Proc. Int’l Conf. Visual Communications and Imagehad a 640 × 480 spatial resolution. In this case, Processing, Int’l Soc. for Optonics and Photonicswe obtained equivalent vehicle detection results (SPIE), 2004, pp. 881–892.(see 5. N. McFarlane and C. Schofield, “Segmentation and There’s a lack of traffic videos available Tracking of Piglets in Images,” Machine Vision andthrough the literature to compare different al- Applications, vol. 8, no. 3, 1995, pp. 187–193.gorithms performing automatic vehicle count- 6. K. Huang et al., “A Real-Time Object Detectinging. Typically, each paper provides its own and Tracking System for Outdoor Night Surveil-results on specific videos that aren’t available lance,” Pattern Recognition, vol. 41, no. 1, 2008,for other researchers. Moreover, a proposed pp. 432–444.algorithm’s accuracy is usually determined bycomparing, on each video-sequence, the visual Angel Sánchez is an associate professor in the De-inspection count with the automatic count that partment of Computing at Rey Juan Carlos Univer-each algorithm provides. Our two original sity, Madrid. His research interests include computervideo sequences, along with the correspond- vision applications, pattern recognition, biometrics,ing analyzed and annotated videos and the de- and soft computing techniques. Sanchez has a PhDtection results are available at www.etsii.urjc. in computer science from the Technical University ofes/~ansanche/traffic_Demo. Our hope is that Madrid. He is a member of IEEE Computer Society andother researchers will use these two videos for the Spanish Association of Pattern Recognition andcomparison purposes. Image Analysis. Contact him at s part of our future work, we plan to Pedro D. Suárez is a doctoral student at Rey Juan test the proposed system’s robustness Carlos University, Madrid. His research interests under more difficult illumination (such include computer vision and pattern recognition. as in night traffic videos) and weather Suarez has an MS in computer science from Rey Juanconditions (such as when snow accumulates Carlos University at Madrid (Spain). Contact him ataround roads). We also plan to test vehicle detec- and tracking at night.6 Another interestingimprovement would be to detect and analyze the Aura Conci is a professor in the Computer Scienceevolution of traffic congestions in particular road Department at Fluminense Federal University, Rioregions over time. de Janeiro. Her research interests include computer vision and image processing. Conci has a DSc in civilAcknowledgments engineering from Pontifical Catholic University in RioThe Spanish project TIN2008-06890-C02-02 partially de Janeiro. Contact her at this work, as did the Brazilian NationalResearch Council (CNPq) and Brazilian Coordination Éldman de Oliveira Nunes is an instructor at thefor the Improvement of Higher Education Personnel School of Complementary Training at the Army(CAPES) program. School of Management, Salvador, Brazil. His research interests include computer vision and image process-References ing. Nunes has a DSc in computer science from Flu-1. V. Kastrinaki, M. Zervakis, and K. Kalaitzakis, “A minense Federal University in Rio de Janeiro. Contact Survey of Video Processing Techniques for Traffic him at 2011 45