IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011                                                     ...
2468                                                                        IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 5...
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011                                                     ...
2470                                                                                              IEEE TRANSACTIONS ON AUT...
IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011                                                     ...
2472                                                                             IEEE TRANSACTIONS ON AUTOMATIC CONTROL, V...
Upcoming SlideShare
Loading in …5

Video surveillance over wireless sensor and actuator networks using active cameras


Published on

services on......
If ur intrested in these project please feel free to contact us@09640648777,Mallikarjun.V

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Video surveillance over wireless sensor and actuator networks using active cameras

  1. 1. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011 2467in the entire network. Since a minimum load imbalance does not nec- Video Surveillance Over Wireless Sensor andessarily lead to a minimum total service load, this optimization goal Actuator Networks Using Active Camerasrequires some compromise between the two minimization criteria. Dalei Wu, Song Ci, Senior Member, IEEE, REFERENCES Haiyan Luo, Student Member, IEEE, Yun Ye, and Haohong Wang, Member, IEEE [1] I. F. Akyildiz and I. H. Kasimoglu, “Wireless sensor and actor net- works,” Ad Hoc Networks J., vol. 2, pp. 351–367, 2004. [2] A. Nayak and I. Stojmenovic, Wireless Sensor and Actuator Networks: Algorithms and Protocols for Scalable Coordination and Data Com- Abstract—Although there has been much work focused on the camera munication. New York: Wiley, 2010. control issue on keeping tracking a target of interest, few has been done [3] Y. Mei, Y.-H. Lu, Y. C. Hu, and C. S. G. Lee, “Reducing the number on jointly considering the video coding, video transmission, and camera of mobile sensors for coverage tasks,” in Proc. IEEE/RSJ IROS, 2005, control for effective and efficient video surveillance over wireless sensor pp. 1426–1431. and actuator networks (WSAN). In this work, we propose a framework for [4] Y. Paschalidis and P. Pennesi, “A distributed actor-critic algorithm and real-time video surveillance with pan-tilt cameras where the video coding applications to mobile sensor network coordination problems,” IEEE and transmission as well as the automated camera control are jointly op- Trans. Autom. Control, vol. 52, no. 2, pp. 492–497, Feb. 2010. timized by taking into account the surveillance video quality requirement [5] P. Corke, R. Peterson, and D. Rus, “Localization and navigation as- and the resource constraint of WSANs. The main contributions of this work sisted by networked cooperating sensors and robots,” Int. J. Robot. are: i) an automated camera control method is developed for moving target Res., vol. 24, no. 9, pp. 771–786, 2005. tracking based on the received surveillance video clip in consideration of [6] A. Panousopoulou, E. Kolyvas, Y. Koveos, A. Tzes, and J. Lygeros, the impact of video transmission delay on camera control decision making; “Robot-assisted target localization using a cooperative scheme re- ii) a content-aware video coding and transmission scheme is investigated to lying on wireless sensor networks,” in Proc. IEEE CDC, 2006, pp. save network node resource and maximize the received video quality under 1031–1036. the delay constraint of moving target monitoring. Both theoretical and ex- perimental results demonstrate the superior performance of the proposed [7] X. Li, N. Santoro, and I. Stojmenovic, “Localized distance-sensitive optimization framework over existing systems. service discovery in wireless sensor and actor networks,” IEEE Trans. Computers, vol. 58, no. 9, pp. 1275–1288, Sep. 2009. Index Terms—Camera control, content-aware video coding and trans- [8] Y. Mei, C. Xian, S. Das, Y. C. Hu, and Y.-H. Lu, “Replacing failed mission, video tracking, wireless sensor networks. sensor nodes by mobile robots,” in Proc. Workshop Wireless Ad Hoc Sensor Networks (WWASN), 2006, p. 87. [9] R. C. Shah, S. Roy, S. Jain, and W. Brunette, “Data mules: Modeling I. INTRODUCTION a three-tier architecture for sparse sensor networks,” Ad Hoc Netw. J., vol. 1, no. 2–3, pp. 215–233, 2003. Recently, as one type of the most popular wireless sensor and [10] W. Zhao, M. Ammar, and E. Zegura, “Controlling the mobility of mul- actuator networks (WSANs) [1], [2], wireless video sensor networks tiple data transport ferries in a delay-tolerant network,” in Proc. IEEE with active cameras have attracted a lot of research attentions and INFOCOM, 2005, pp. 1407–1418. been adopted for various applications, such as intelligent transporta- [11] R. Sugihara and R. K. Gupta, “Optimal speed control of mobile node for data collection in sensor networks,” IEEE Trans. Mobile tion, environmental monitoring, homeland security, construction site Computing, vol. 9, no. 1, pp. 127–139, Jan. 2010. monitoring, and public safety. Although significant amount of work [12] G. Xing, T. Wang, W. Jia, and M. Li, “Rendezvous design algorithms has been done on wireless video surveillance in literature, major chal- for wireless sensor networks with a mobile base station,” in Proc. ACM lenges still exist in transmitting videos over WSANs and automatically MobiHoc, 2008, pp. 231–240. controlling cameras due to the fundamental limits of WSANs, such [13] M. Ma and Y. Yang, “Data gathering in wireless sensor networks with as, limitations on computation, memory, battery life, and network mobile collectors,” in Proc. IEEE IPDPS, Miami, FL, Apr. 2008, pp. 1–9. bandwidth at sensors, as well limitations on actuating speed, delay, [14] T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction and range at actuators. to Algorithms. Cambridge, MA: MIT Press, 2001. Some work has been focused on automated camera control for video [15] S. Nesamony, M. K. Vairamuthu, M. E. Orlowska, and S. W. Sadiq, surveillance applications. In [3], an algorithm was proposed to provide “On Optimal Route Computation of Mobile Sink in a Wireless Sensor automated control of a pan-tilt camera by using the captured visual Network,” ITEE, Univ. Queensland, Brisbane, Australia, Tech. Rep. information only to follow a person’s face and keep the face image 465, 2006. [16] A. Dumitrescu and J. S. B. Mitchell, “Approximation algorithms for centered in the camera view. In [4], an image-based pan-tilt camera TSP with neighborhoods in the plane,” in Proc. ACM-SIAM SODA, control method was proposed for automated surveillance systems with 2001, pp. 38–46. multiple cameras. The work in [5] focused on the control of a set of [17] F. Aurenhammer, “Voronoi diagrams—A survey of a fundamental geo- pan-tilt-zoom (PTZ) cameras for acquiring closeup views of subjects metric data structure,” ACM Comput. Surv., vol. 23, no. 3, pp. 345–405, 1991. [18] J. Kirk, Traveling Salesman Problem—Genetic Algorithm [On- Manuscript received April 03, 2010; revised March 18, 2011; accepted July line]. Available: 09, 2011. Date of publication August 08, 2011; date of current version October change/13680 05, 2011. This work was supported in part by the National Science Foundation (NSF) under Grant CCF-0830493. Recommended by Associate Editor I. Stoj- menovic. D. Wu, S. Ci, and Y. Ye are with the Department of Computer and Electronics Engineering, University of Nebraska-Lincoln, Omaha, NE 68182 USA (e-mail:;; H. Luo is with Cisco Systems, Austin, TX 79729 USA (e-mail: H. Wang is with TCL Corporation, Santa Clara, CA 95054 USA (e-mail: Color versions of one or more of the figures in this technical note are available online at Digital Object Identifier 10.1109/TAC.2011.2164034 0018-9286/$26.00 © 2011 IEEE
  2. 2. 2468 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011Fig. 1. System model of the proposed framework for real-time video surveillance over WSANs with active cameras.based on the output of a set of fixed cameras. To extend the applications II. PROBLEM DESCRIPTIONof these camera control methods into video surveillance over WSANs,both video transmission over WSANs and its impact on the accuracy A. Proposed System Modelof camera control need to be considered. Fig. 1 shows the proposed system model for wireless video surveil- It is challenging to transmit videos over resource-constrained lance. The captured videos are coded at the camera, and the resultingWSANs. Low network bandwidth results in large delay in video de- video packets are transmitted over a wireless sensor network to a RCUlivery, which may not satisfy the low latency requirements in real-time where videos are reconstructed for target monitoring. To keep trackingtarget detection and camera control decision making at the surveillance the target, we propose that the camera control decision-making be per-center. Also, lossy network channels lead to packet losses during video formed at the RCU since RCU has more computational capability andtransmission. The resulting received video distortion may not satisfy more information about the target. Therefore, based on the video anal-the quality requirement of target observation at the surveillance center. ysis at the RCU, the controller needs to calculate and determine theMoreover, large video distortion may affect the accuracy of camera camera control parameters, i.e., pan and tilt velocities, and send themcontrol since camera control decision is made based on the received back to the camera motor.videos. Finally, on the one hand, each node in WSNAs is powered by Note that each captured video frame can be considered as being com-either batteries or solar-energy-harvesting devices, meaning that power posed of a target part and a background part. To save node resource, inis of utmost importance, and must be aggressively conserved. On the this work we aim to develop a content-aware approach to the videoother hand, wireless video surveillance is extremely useful to provide processing by which the target part is coded and transmitted with acontinuous visual coverage in the surveillance areas. As a result, higher priority than the background part. In other words, the limitedthe continuous transmission of large amount of surveillance videos network resources are concentrated on the coding and transmission ofposes significant challenges to the power conservation of WSANs. the target part while the background part is processed in a best-effortTill now, although there has been work done on resource allocation fashion. Note that in this work the motivation for background-fore-for co-design of communication protocols and control strategies [6], ground separation is mainly to roughly divide a captured video framefew existing work focuses on the joint optimization of video coding, into different regions and find the region of interest (ROI, i.e., the targetvideo transmission, and camera control for real-time video monitoring part) for resource allocation adaptation by content-aware video codingapplications in WSANs. and transmission rather than to accurately perform target identification In this technical note, we propose a framework for real-time video and extraction. Therefore, suitable background-foreground separationsurveillance over WSANs in which pan/tilt cameras are used to track methods [11]–[14] can be selected to achieve a good tradeoff betweenmoving targets of interest and transmit the captured videos to a remote the accuracy and complexity at the camera side based on the camera’scontrol unit (RCU). The main contributions of this technical note in- computational capability.clude: (i) an automated camera control method is developed to keeptracking the target by taking into account the impacts of the delay of B. Kinematic Model of Pan-Tilt Camerasvideo transmission and control decision feedback on the control-deci-sion making process; (ii) a content-aware video coding and transmis- The control objective is to maintain the target being tracked in thesion scheme is investigated to save network node resource while max- center of the camera view. In the following, we adopt the kinematicimizing the received video quality under the delay constraint of target model of pan-tilt cameras developed in [3] to explain the camera controlmonitoring. It is worth noting that although the camera control deci- objective. Let (xc ; yc ) be the target offset with respect to the center ofsion-making is performed by the central RCU, the proposed video pro- the image, i.e., the coordinates of the centroid of the segmented targetcessing method can also be used in smart cameras [7] or mobile agents image inside the window where the target is located, as shown in Fig. 2.[8] in distributed video surveillance systems [9], [10]. We next take the pan movement of the camera as an example to study The remainder of this technical note is organized as follows. In the camera control problem.Section II, we present the system model and formulated optimization Let f be the camera focal length. As shown in Fig. 3, the relationshipproblem. Section III presents an automated camera control method. between target coordinate xc and pan angle can be expressed asThe solution procedure is described in Section IV. Section V showsexperimental results and Section VI concludes this technical note. xc = f tan( 0 ) (1)
  3. 3. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011 2469 optimally estimate the overall distortion of decoder frame reconstruc- tion due to quantization, error propagation, and error concealment by recursively computing the total decoder distortion at pixel level precision. Moreover, in consideration of the importance difference for video surveillance between the target and background parts, we assume that each packet i is associated with a quality impact factor i . i can be determined by the quality requirement on the corresponding video parts. Given i , the expected distortion of the whole video frame, denoted by E [D], can be written as [15] E [D] = I=1 i E [Di ](si ; pi ; ). Here E [Di ] is the expected distor- i tion of packet i , which is a function of the source coding parameter si , end-to-end packet loss rate pi , and error concealment scheme forFig. 2. Target offset in the image plane. the corresponding GOB [16]. The calculation of pi will be presented in Section IV-B. 3) Problem Formulation of Content-Aware Video Coding and Trans- mission: The objective of video coding and transmission optimiza- tion is to determine the optimal values of source coding parameter and channel transmission parameter vectors fsi ; ci g for each packet i of the current video frame to maximize the expected received video quality under the delay constraint, i.e. I fsi ; ci g = arg min i E [Di ] i=1 s:t: : Ti (si ; ci ) T max (i = 1; 2; 1 1 1 ; I ): (3) Note that the optimization is performed one frame at a time. Nonetheless, this framework can potentially be improved by opti-Fig. 3. Dependence of the target coordinate x on the pan angle . mizing the video encoding and transmission over multiple buffered frames, which can integrate the packet dependencies caused by both error concealment and prediction in source coding into the optimiza-where is the angle of the target centroid in the frame FXY. Differen- tion framework. Such a scheme, however, would lead to a considerablytiating (1) with respect to time t, a kinematic model for the xc offset is higher computational complexity. The detailed solution to the problemobtained as in (3) will be discussed in Section IV. x_c = f (! 0 ! ) 0 ) 2 (2) cos ( III. CAMERA CONTROL DECISION MAKING AT THE RCU In this section, we modify and improve an existing pan-tilt camerawhere the pan velocity ! is considered as a control input. The angular control algorithm [3] by taking into account the impacts of both videovelocity ! of the target depends on the instantaneous movement of transmission delay and control decision feedback delay on the camerathe target with respect to the frame FXY. The control problem consists control decision making process at the RCU. In the following, we takein finding an adaptive feedback control law for the system as shown in(2) with control input ! such that limt !1 xc (t) = 0. the pan angle control process as an example to explain the improved algorithm.C. Proposed Content-Aware Video Coding and Transmission As discussed in Section II-B, the objective of pan angle control is to determine control input ! such that limt !1 xc (t) = 0. Similar to Without loss of generality, it is assumed that the target and back- the work in [3] we use the following control law:ground parts in the current video frame are composed of Io and Ibgroups of blocks (GOB), respectively. We also assume that each GOB ! = ! + xc ^ (4)is encoded into a packet. Let f1 ; 2 ; 1 1 1 ; I g be the resulting Iotarget packets and fI +1 ; I +2 ; 1 1 1 ; I +I g the Ib background where is a positive gain, and the estimate ! of ! is obtained from ^packets of the current video frame. For simplicity, define I = Io + Ib . the dynamic part of the automated control, which is designed as a pa-For each packet i , let si be the corresponding source coding param- rameter update law. According to Proposition 1 in [3], to ensure thateter vector, and ci the channel transmission parameter vector. the resulting closed-loop adaptive camera control system is asymptot- 1) Delay Constraint: To perform accurate camera control and pro- ically stable, the update law is derived asvide smooth video playback at the RCU, each video frame to be trans- ! = fxc :mitted is associated with a frame decoding deadline T max . In real- _ (5) 0 ) ^ cos ( 2time video communications, the average video frame decoding dead-line T max is linked with the video frame rate fr as T max (1=fr ) Both target offset fxc ; yc g and pan/tilt angle f; g needed for con-[15]. The frame decoding deadline indicates that a delay deadline is trol decision making are measured based on the received video frames.imposed on the transmission of each packet composing the frame by Let Fk be the video frame captured at discrete time instant tk (k =T max [15], i.e., Ti (si ; ci ) T max (i = 1; 2; 1 1 1 ; I ), where Ti (si ; ci ) 0; 1; 2; 1 1 1). From Fk the offsets fxct ; yct g and the pan and tilt an-is the end-to-end delay of packet i transmitted from the camera to the gles ft ; t g can be measured at the RCU by using the Mean ShiftRCU, which is a function of si and ci . algorithm as in [3]. Note that the motivation for conducting the Mean 2) Expected Received Video Quality: We employ the ROPE Shift algorithm at the RCU instead of at the camera side is based on thealgorithm [16] to evaluate the received video distortion. ROPE can observation that in some scenarios the RCU may already have some a
  4. 4. 2470 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011 TABLE I IV. OPTIMIZED CONTENT-AWARE VIDEO CODING AND TRANSMISSION MODIFIED CAMERA CONTROL ALGORITHM AT THE RCU A. Content-Aware Video Coding Let si 2 be the source coding parameters for the ith GOB, where is the set of all admissible values of si and j j = J . For a given GOB, different coding parameters will lead to different amounts of compres- sion-induced distortion. On the other hand, different coding parameters also result in different packet lengths and packet loss rates, which will lead to different amounts of transmission-induced distortion. A robust error concealment technique helps avoid significant visible error due to the packet loss in the reconstructed frames at the decoder. However,priori knowledge of the target, such as the color or size information some error concealment strategies cause packet dependencies, whichof the target. It is expected that integrating these information into the makes the source coding optimization further complicated. It is impor-Mean Shift algorithm could provide more accurate measurement of the tant to point out that, although some straightforward error concealmentoffsets and quantities of the target. strategies do not cause packet dependencies, as a generic framework, Note that fxct ; yct g and ft ; t g only represent the position the more complicated scenario is considered here as a superset for theinformation of the target and camera at time instant t = tk when video simpler cases.frame Fk is captured. In fact, a significant amount of delay is incurred Without loss of generality, we assume that the current GOB dependsduring transmitting the frame Fk to the RCU as well as sending the on its previous z GOBs (z 0). Therefore, the optimization goal incontrol decision back to the camera. Therefore, in order to perform (3) becomesaccurate camera control, the further change of the target position dueto the possible movement of the target during this delay period also Ineeds to be considered in camera control decision making. min E [Di ](si0z ; si0z+1 ; 1 1 1 ; si ); i0z 0 (12) ^ At t = t0 , the first estimate for ! , !t , is set arbitrary. For the s ;111;s i=1control law (4) at t = tk , we modify the relation !t = !t + xct ^in [3] into where E [Di ](si0z ; si0z+1 ; 1 1 1 ; si ) represents the expected distor- tion of the ith GOB, which depends on the z + 1 decision vectors !t +T =! ^ t +T + x 0 ct +T (6) fsi0z ; si0z+1 ; 1 1 1 ; si g under the packet delay deadline. The problem (12) can be solved by a dynamic programming approach. Due to thewhere x0 +T is the estimate of offset xc when the camera control ct limit of space, we will not elaborate the solution procedure in thisdecision arrives at the camera, and Tk is the total delay of the video technical note. Interested readers may refer to [15]. The computational complexity of the above algorithm is O(I 2j jz ),frame transmission and the control decision feedback. Next we consider how to calculate x0 +T based on measurement ctfxct ; t ; t g by taking into account Tk . Let Tdk be the end-to-end which is much more efficient than the exponential computational com-delay in transmitting video frame Fk from the camera to the RCU, plexity of an exhaustive search algorithm. Clearly for cases withwhich will be discussed in Section IV-B. Let Tf k be the delay in smaller z , the complexity is quite practical to perform the optimiza-sending back the control decision based on Fk from the RCU to the tion. On the other hand, for larger z , the complexity can be limitedcamera. Without loss of generality, we assume that Tf k is a constant. by reducing the cardinality of . The practical solution would be anTherefore, Tk = Tdk + Tf k . Based on fxct ; t ; t g, we can engineering decision and tradeoff between the camera’s computational ^ ^compute both estimate t +T of and estimate t +T of at time capability and the optimality of the solution. For resource-constrainedinstant tk + Tk as WSANs, simple error concealment strategies could be considered to further reduce the computational complexity. ^ t +T = + ! t t 1 Tk ; (7) ^ t +T = + ! ^ t t 1 Tk (8) B. Content-Aware Video Transmission Let cg denote the video packet class where g = 0 if packet i jwhere t can be derived from (1) as is a target packet, and g = 1 if it is a background packet. Without loss of generality, we assume that (u; v ) is a link/hop of path R, t = + arctan xf t ct : (9) where nodes u and v are the hth and (h + 1)th nodes of path R,Based on (1), estimate x0 +T of offset xc at time instant tk + Tk can respectively. In wireless environments, the transmission of a packet ct over each hop (u; v ) of the network can be modeled as follows:be calculated as once the packet arrives at node u, the packet header is extracted to x0 ct +T = f 1 tan ^ t +T ^ 0 t +T : (10) determine the packet class. If it is a target packet, it will be put ahead of all background packets in the queue of node u, waiting for At t = tk+1 + Tk+1 , to derive the control input !t +T , the being transmitted/served over link (u; v ). We assume that all packets ^estimate !t +T of !t +T can be calculated based on (5) of the same class in the queue of node u are scheduled followingas follows: a first-come, first-served (FCFS) fashion. If the packet gets lost during transmission with the packet error probability pg;(u;v ) due to ^ !t =! ^ + fx0 ct +T : signal fading over the link (u; v ), it will be retransmitted until it is +T t +T cos ^2 t +T ^ 0 t +T (11) either successfully received or discarded because its delay deadline T max was exceeded. As a result of the retransmission mechanism,The complete modified camera control algorithm at the RCU is sum- the packet loss probability over link (u; v ) is mainly exhibited asmarized in Table I. the probability of packet drop pg;u due to delay deadline expiration
  5. 5. IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011 2471 TABLE II PROCESSING LOAD DISTRIBUTION OF THE PROPOSED FRAMEWORKwhen queueing at node u. Based on priority queueing analysis, pg;ucan be calculated as [14] pg;u = Prob E Wg;(u;v) + tg;u T 0 max 1 = g;u E [Zg;u ] g=0 1 T max 0 tg;u 0 g;u E [Zg;u ] 2 exp 0 g =0 E Wg;(u;v) (13)where tg;u is the arrival time of the packets of class cg at node 0u, g;u the average arrival rate of the Poisson input traffic into thequeue at node u, E [Zg;u ] the average service time at node u, andE [Wg;(u;v) ] the average packet waiting time at the queue of node u. From (13), the end-to-end packet loss probability of a packet ofvideo class cg transmitted over path R can be expressed as pg =10 (u;v)2R (10 pg;u). The end-to-end packet delay Tdk is the sum ofthe packet delay tg;(u;v) over each link (u; v ), which can be obtained as[15] Tdk = (u;v)2R tg;(u;v) = (u;v)2R fE [Wg;(u;v) ]+E [Zg;u ]g. V. EXPERIMENTAL RESULTS Experiments of video surveillance were conducted in the lobby ofa building to evaluate the performance of the proposed framework interms of the camera control effectiveness and the surveillance videoquality. All captured video frames except the first one were coded asinter frames. Each video frame was divided into a target part and a back-ground part by the method of Mixture of Gaussians. The target andbackground parts were separately coded. The resulting video streamwas transmitted to a server via a wireless sensor network. Based onthe received video, the target offset was calculated at the server byusing the Mean Shift algorithm for camera-control decision making.The processing load distribution of the proposed framework is pre-sented in Table II, where Nf is the video frame size, m is the numberof Gaussian distribution and practically set to be between 3 and 5 [12],c is usually between 10 and 20 [16], depending on both the frametypes (intra-coded or inter-coded) and the adopted error concealmentschemes, and Nw is the search window size. In our first set of experiments, the target first made circular move- Fig. 4. Evaluation of the camera control effectiveness. (a) Pan velocity andment to the left at an angular velocity of ! = 0:4 rad=second for estimated angular velocity of the target. (b) Pan angle and target angle. (c) Target20 seconds, and then made circular movement to the right at ! = offset.00:3 rad=second for another 20 seconds. Initially, the angle =0:5 rad, offset xct = 0:02 m, and pan angle t = 0 rad. Fig. 4(a)shows that both the estimated angular velocity of the target and the in Fig. 5, the proposed control strategy is more effective, leading toangular velocity of pan tend asymptotically to the actual angular ve- smaller target offset xc with respect to the center of the image. This islocity of the target. Fig. 4(b) shows that the pan and target angles con- because the proposed strategy takes into account the target movementverge after some time duration under each movement scenario. Fig. 4(c) during the delay time of video transmission and control decision feed-shows that the offset xc tends to be zero after some time duration under back.each movement scenario, meaning that the captured target is at the We evaluated the video quality obtained by the proposed content-center of the image plane. Therefore, Fig. 4(a)–(c) demonstrate the ef- aware video coding and transmission. Under each movement scenariofectiveness of the proposed camera control method. of the target, the surveillance videos were captured at two different In the second set of experiments, the target was allowed to walk back video frame rates: 15 and 30 f/s. Fig. 6(a) and (b) show the receivedand forth across the lobby. We compared the proposed camera control video quality of different video parts at the RCU, measured by peakstrategy with the control strategy without considering the impact of the signal-to-noise ratio (PSNR). We observe that the PSNRs of the targetdelay of video transmission and control decision feedback. As shown part are over 5 dB higher than those of the background part under both
  6. 6. 2472 IEEE TRANSACTIONS ON AUTOMATIC CONTROL, VOL. 56, NO. 10, OCTOBER 2011 content-aware video coding and transmission scheme was developed to maximize the received video quality under the delay constraint imposed by video surveillance applications. The impact of the delay of video transmission and control decision feedback on camera-control decision making has also been investigated to provide accurate camera control. Experimental results demonstrate the efficacy of the proposed co-design framework. REFERENCES [1] I. F. Akyildiz and I. H. Kasimoglu, “Wireless sensor and actor net- works: Research challenges,” Ad Hoc Netw., vol. 2, no. 4, pp. 351–367, May 2004. [2] X. Cao, J. Chen, Y. Xiao, and Y. Sun, “Building environment con- trol with wireless sensor and actuator networks: Centralized versus dis- tributed,” IEEE Trans. Ind. Electron., vol. 27, no. 11, pp. 3596–3605,Fig. 5. Comparison of different camera control strategies. Nov. 2010. [3] P. Petrov, O. Boumbarov, and K. Muratovski, “Face detection and tracking with an active camera,” in Proc. 4th Int. IEEE Conf. Intell. Syst., Varna, Bulgaria, Sep. 2008, pp. 14–39. [4] S. Lim, A. Elgammal, and L. S. Davis, “Image-based pan-tilt camera control in a multi-camera surveillance environment,” in Proc. Int. Conf. Mach. Intell., 2003, pp. 645–648. [5] N. Krahnstoever, T. Yu, S. Lim, and K. Patwardhan, “Collaborative control of active cameras in large-scale surveillance,” in Multi-Camera Networks—Principles and Applications, H. Aghajan and A. Cavallaro, Eds. Amsterdam, The Netherlands: Elsevier, 2009. [6] A. Deshpande, C. Guestrin, and S. Madden, “Resource-aware wireless sensor-actuator networks,” IEEE Data Eng., vol. 28, no. 1, pp. 40–47, 2005. [7] Z. Zivkovic and R. Kleihorst, “Smart cameras for wireless camera networks: Architecture overview,” in Multi-Camera Networks—Prin- ciples and Applications, H. Aghajan and A. Cavallaro, Eds. Ams- terdam, The Netherlands: Elsevier, 2009. [8] B. Rinner and M. Quaritsch, “Embedded middleware for smart camera networks and sensor fusion,” in Multi-Camera Networks—Principles and Applications, H. Aghajan and A. Cavallaro, Eds. Amsterdam, The Netherlands: Elsevier, 2009. [9] M. Dalai and R. Leonardi, “Video compression for camera networks: A distributed approach,” in Multi-Camera Networks—Principles and Applications, H. Aghajan and A. Cavallaro, Eds. Amsterdam, The Netherlands: Elsevier, 2009. [10] P. Dragotti, “Distributed compression in multi-camera systems,” in Multi-Camera Networks—Principles and Applications, H. AghajanFig. 6. Comparison of the received surveillance video quality. (a) Average and A. Cavallaro, Eds. Amsterdam, The Netherlands: Elsevier, 2009.PSNRs when = 0 4 rad : . (b) Average PSNRs when = 00 3 rad : . (c) Re- [11] M. Piccardi, “Background subtraction techniques: A review,” in Proc.ceived video frame by using the proposed content-aware video coding and trans- IEEE Int. Conf. Syst., Man Cybern., 2004, pp. 3099–3104.mission. (d) Received video frame without using content-aware video coding [12] C. Stauffer and W. Grimson, “Learning patterns of activity using real-and transmission. time tracking,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 8, pp. 747–757, 2000. [13] Y. Sheikh, O. Javed, and T. Kanade, “Background Subtraction for Freely Moving Cameras,” in Proc. IEEE Int. Conf. Comp. Vision,video frame rates. Moreover, the PSNRs of the target are also much 2009, pp. 1219–1225.higher than those of the received videos without using content-aware [14] D. Wu, H. Luo, S. Ci, H. Wang, and A. Katsaggelos, “Quality-driven optimization for content-aware real-time video streaming in wirelesscoding and transmission. mesh networks,” in Proc. IEEE GLOBECOM, Dec. 2008, pp. 1–5. Fig. 6(c) and (d) show the comparison of the received surveillance [15] D. Wu, S. Ci, H. Luo, H. Wang, and A. Katsaggelos, “Applica-video quality with and without the proposed content-aware video tion-centric routing for video streaming over multi-hop wirelesscoding and transmission scheme. As shown in Fig. 6(c), the target part networks,” IEEE Trans. Circuits Syst. Video Technol., vol. 20, no. 12, pp. 1721–1734, Dec. 2010.has a much better visual quality than the background part. Moreover, [16] R. Zhang, S. L. Regunathan, and K. Rose, “Video coding with op-the target part as shown in Fig. 6(c) also has a better visual quality timal inter/intra-mode switching for packet loss resilience,” IEEE J.than the received video without using the content-aware video coding Sel. Areas Commun., vol. 18, no. 6, pp. 966–976, Jun. 2000.and transmission scheme as shown in Fig. 6(d). VI. CONCLUSION In this technical note, we presented a co-design framework for videosurveillance with active cameras by jointly considering the cameracontrol strategy and video coding and transmission over wirelesssensor and actuator networks. To keep tracking a target of interest,an automated camera control method was proposed based on thereceived surveillance videos. To save the network node resource, a