Computer Networks 51 (2007) 921–960 www.elsevier.com/locate/comnet A survey on wireless multimedia sensor networks Ian F. Akyildiz *, Tommaso Melodia, Kaushik R. Chowdhury Broadband and Wireless Networking Laboratory, School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332, United States Received 11 March 2006; received in revised form 6 August 2006; accepted 5 October 2006 Available online 2 November 2006Abstract The availability of low-cost hardware such as CMOS cameras and microphones has fostered the development of Wire-less Multimedia Sensor Networks (WMSNs), i.e., networks of wirelessly interconnected devices that are able to ubiqui-tously retrieve multimedia content such as video and audio streams, still images, and scalar sensor data from theenvironment. In this paper, the state of the art in algorithms, protocols, and hardware for wireless multimedia sensor net-works is surveyed, and open research issues are discussed in detail. Architectures for WMSNs are explored, along withtheir advantages and drawbacks. Currently oﬀ-the-shelf hardware as well as available research prototypes for WMSNsare listed and classiﬁed. Existing solutions and open research issues at the application, transport, network, link, and phys-ical layers of the communication protocol stack are investigated, along with possible cross-layer synergies andoptimizations.Ó 2006 Elsevier B.V. All rights reserved.Keywords: Wireless sensor networks; Multimedia communications; Distributed smart cameras; Video sensor networks; Energy-awareprotocol design; Cross-layer protocol design; Quality of service1. Introduction vesting information from the physical environment, performing simple processing on the extracted data Wireless sensor networks (WSN)  have drawn and transmitting it to remote locations. Signiﬁcantthe attention of the research community in the last results in this area over the last few years have ush-few years, driven by a wealth of theoretical and ered in a surge of civil and military applications. Aspractical challenges. This growing interest can be of today, most deployed wireless sensor networkslargely attributed to new applications enabled by measure scalar physical phenomena like tempera-large-scale networks of small devices capable of har- ture, pressure, humidity, or location of objects. In general, most of the applications have low band- * width demands, and are usually delay tolerant. Corresponding author. Tel.: +1 404 894 5141; fax: +1 404 894 More recently, the availability of inexpensive7883. E-mail addresses: firstname.lastname@example.org (I.F. Akyildiz), tomma- hardware such as CMOS cameras and email@example.com (T. Melodia), firstname.lastname@example.org that are able to ubiquitously capture multimedia(K.R. Chowdhury). content from the environment has fostered the1389-1286/$ - see front matter Ó 2006 Elsevier B.V. All rights reserved.doi:10.1016/j.comnet.2006.10.002
922 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960development of Wireless Multimedia Sensor Net- to identify the violator, or buﬀer images andworks (WMSNs) [54,90], i.e., networks of wirelessly streams in case of accidents for subsequent acci-interconnected devices that allow retrieving video dent scene analysis.and audio streams, still images, and scalar sensor • Advanced health care delivery. Telemedicine sen-data. With rapid improvements and miniaturization sor networks  can be integrated with 3G mul-in hardware, a single sensor device can be equipped timedia networks to provide ubiquitous healthwith audio and visual information collection mod- care services. Patients will carry medical sensorsules. As an example, the Cyclops image capturing to monitor parameters such as body temperature,and inference module , is designed for extre- blood pressure, pulse oximetry, ECG, breathingmely light-weight imaging and can be interfaced activity. Furthermore, remote medical centerswith a host mote such as Crossbow’s MICA2  will perform advanced remote monitoring ofor MICAz . In addition to the ability to retrieve their patients via video and audio sensors, loca-multimedia data, WMSNs will also be able to store, tion sensors, motion or activity sensors, whichprocess in real-time, correlate and fuse multimedia can also be embedded in wrist devices .data originated from heterogeneous sources. • Automated assistance for the elderly and family Wireless multimedia sensor networks will not monitors. Multimedia sensor networks can beonly enhance existing sensor network applications used to monitor and study the behavior of elderlysuch as tracking, home automation, and environ- people as a means to identify the causes ofmental monitoring, but they will also enable several illnesses that aﬀect them such as dementia .new applications such as: Networks of wearable or video and audio sensors can infer emergency situations and immediately• Multimedia surveillance sensor networks. Wireless connect elderly patients with remote assistance video sensor networks will be composed of inter- services or with relatives. connected, battery-powered miniature video • Environmental monitoring. Several projects on cameras, each packaged with a low-power wire- habitat monitoring that use acoustic and video less transceiver that is capable of processing, feeds are being envisaged, in which information sending, and receiving data. Video and audio has to be conveyed in a time-critical fashion. sensors will be used to enhance and complement For example, arrays of video sensors are already existing surveillance systems against crime and used by oceanographers to determine the evolu- terrorist attacks. Large-scale networks of video tion of sandbars via image processing techniques sensors can extend the ability of law enforcement . agencies to monitor areas, public events, private • Person locator services. Multimedia content such properties and borders. as video streams and still images, along with• Storage of potentially relevant activities. Multime- advanced signal processing techniques, can be dia sensors could infer and record potentially rel- used to locate missing persons, or identify crimi- evant activities (thefts, car accidents, traﬃc nals or terrorists. violations), and make video/audio streams or • Industrial process control. Multimedia content reports available for future query. such as imaging, temperature, or pressure• Traﬃc avoidance, enforcement and control sys- amongst others, may be used for time-critical tems. It will be possible to monitor car traﬃc in industrial process control. Machine vision is the big cities or highways and deploy services that application of computer vision techniques to oﬀer traﬃc routing advice to avoid congestion. industry and manufacturing, where information In addition, smart parking advice systems based can be extracted and analyzed by WMSNs to on WMSNs  will allow monitoring available support a manufacturing process such as those parking spaces and provide drivers with auto- used in semiconductor chips, automobiles, food mated parking advice, thus improving mobility or pharmaceutical products. For example, in in urban areas. Moreover, multimedia sensors quality control of manufacturing processes, may monitor the ﬂow of vehicular traﬃc on details or ﬁnal products are automatically highways and retrieve aggregate information inspected to ﬁnd defects. In addition, machine such as average speed and number of cars. Sen- vision systems can detect the position and orien- sors could also detect violations and transmit tation of parts of the product to be picked up by video streams to law enforcement agencies a robotic arm. The integration of machine vision
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 923 systems with WMSNs can simplify and add tures such as Diﬀserv and Intserv for Internet QoS ﬂexibility to systems for visual inspections and delivery have been developed. However, there are automated actions that require high-speed, several main peculiarities that make QoS delivery high-magniﬁcation, and continuous operation. of multimedia content in sensor networks an even more challenging, and largely unexplored, task: As observed in , WMSNs will stretch thehorizon of traditional monitoring and surveillance • Resource constraints. Sensor devices are con-systems by: strained in terms of battery, memory, process- ing capability, and achievable data rate .• Enlarging the view. The Field of View (FoV) of a Hence, eﬃcient use of these scarce resources is single ﬁxed camera, or the Field of Regard (FoR) mandatory. of a single moving pan-tilt-zoom (PTZ) camera is • Variable channel capacity. While in wired net- limited. Instead, a distributed system of multiple works the capacity of each link is assumed to cameras and sensors enables perception of the be ﬁxed and pre-determined, in multi-hop wire- environment from multiple disparate viewpoints, less networks, the attainable capacity of each and helps overcoming occlusion eﬀects. wireless link depends on the interference level• Enhancing the view. The redundancy introduced perceived at the receiver. This, in turn, depends by multiple, possibly heterogeneous, overlapped on the interaction of several functionalities that sensors can provide enhanced understanding are distributively handled by all network devices and monitoring of the environment. Overlapped such as power control, routing, and rate policies. cameras can provide diﬀerent views of the same Hence, capacity and delay attainable at each link area or target, while the joint operation of are location dependent, vary continuously, and cameras and audio or infrared sensors can help may be bursty in nature, thus making QoS provi- disambiguate cluttered situations. sioning a challenging task.• Enabling multi-resolution views. Heterogeneous • Cross-layer coupling of functionalities. In multi- media streams with diﬀerent granularity can be hop wireless networks, there is a strict interde- acquired from the same point of view to provide pendence among functions handled at all layers a multi-resolution description of the scene and of the communication stack. Functionalities multiple levels of abstraction. For example, static handled at diﬀerent layers are inherently and medium-resolution camera views can be enriched strictly coupled due to the shared nature of the by views from a zoom camera that provides a wireless communication channel. Hence, the var- high-resolution view of a region of interest. For ious functionalities aimed at QoS provisioning example, such feature could be used to recognize should not be treated separately when eﬃcient people based on their facial characteristics. solutions are sought. • Multimedia in-network processing. Processing of Many of the above applications require the sen- multimedia content has mostly been approachedsor network paradigm to be rethought in view of as a problem isolated from the network-designthe need for mechanisms to deliver multimedia con- problem, with a few exceptions such as jointtent with a certain level of quality of service (QoS). source-channel coding  and channel-adaptiveSince the need to minimize the energy consumption streaming . Hence, research that addressedhas driven most of the research in sensor networks the content delivery aspects has typically not con-so far, mechanisms to eﬃciently deliver application sidered the characteristics of the source contentlevel QoS, and to map these requirements to net- and has primarily studied cross-layer interactionswork layer metrics such as latency and jitter, have among lower layers of the protocol stack. How-not been primary concerns in mainstream research ever, the processing and delivery of multimediaon classical sensor networks. content are not independent and their interaction Conversely, algorithms, protocols and techniques has a major impact on the levels of QoS that canto deliver multimedia content over large-scale net- be delivered. WMSNs will allow performing mul-works have been the focus of intensive research in timedia in-network processing algorithms on thethe last 20 years, especially in ATM wired and wire- raw data. Hence, the QoS required at the applica-less networks. Later, many of the results derived for tion level will be delivered by means of a combi-ATM networks have been readapted, and architec- nation of both cross-layer optimization of the
924 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 communication process, and in-network process- tion and computation with signal processing and ing of raw data streams that describe the phe- several branches of control theory and embedded nomenon of interest from multiple views, with computing. This cross-disciplinary research will diﬀerent media, and on multiple resolutions. enable distributed systems of heterogeneous embed- Hence, it is necessary to develop application- ded devices that sense, interact, and control the independent and self-organizing architectures to physical environment. There are several factors that ﬂexibly perform in-network processing of multi- mainly inﬂuence the design of a WMSN, which are media contents. outlined in this section. Eﬀorts from several research areas will need to • Application-speciﬁc QoS requirements. The wideconverge to develop eﬃcient and ﬂexible WMSNs, variety of applications envisaged on WMSNs willand this in turn, will signiﬁcantly enhance our have diﬀerent requirements. In addition to dataability to interact with the physical environment. delivery modes typical of scalar sensor networks,These include advances in the understanding of multimedia data include snapshot and streamingenergy-constrained wireless communications, and multimedia content. Snapshot-type multimediathe integration of advanced multimedia processing data contain event triggered observations obtainedtechniques in the communication process. Another in a short time period. Streaming multimediacrucial issue is the development of ﬂexible system content is generated over longer time periodsarchitectures and software to allow querying the and requires sustained information delivery.network to specify the required service (thus provid- Hence, a strong foundation is needed in terms ofing abstraction from implementation details). At the hardware and supporting high-level algorithmssame time, it is necessary to provide the service in to deliver QoS and consider application-speciﬁcthe most eﬃcient way, which may be in contrast requirements. These requirements may pertainwith the need for abstraction. to multiple domains and can be expressed, amongst In this paper, we survey the state of the art in others, in terms of a combination of bounds onalgorithms, protocols, and hardware for the devel- energy consumption, delay, reliability, distortion,opment of wireless multimedia sensor networks, or network lifetime.and discuss open research issues in detail. In partic- • High bandwidth demand. Multimedia content,ular, in Section 2 we point out the characteristics of especially video streams, require transmissionwireless multimedia sensor networks, i.e., the major bandwidth that is orders of magnitude higherfactors inﬂuencing their design. In Section 3, we than that supported by currently available sen-suggest possible architectures for WMSNs and sors. For example, the nominal transmission ratedescribe their characterizing features. In Section 4, of state-of-the-art IEEE 802.15.4 compliant com-we discuss and classify existing hardware and proto- ponents such as Crossbow’s  MICAz ortypal implementations for WMSNs, while in Section TelosB  motes is 250 kbit/s. Data rates at least5 we discuss possible advantages and challenges of one order of magnitude higher may be requiredmultimedia in-network processing. In Sections 6– for high-end multimedia sensors, with compara-10 we discuss existing solutions and open research ble power consumption. Hence, high data rateissues at the application, transport, network, link, and low-power consumption transmission tech-and physical layers of the communication stack, niques need to be leveraged. In this respect, therespectively. In Section 11, we discuss cross-layer ultra wide band (UWB) transmission techniquesynergies and possible optimizations, while in seems particularly promising for WMSNs, andSection 12 we discuss additional complementary its applicability is discussed in Section 10.research areas such as actuation, synchronization • Multimedia source coding techniques. Uncom-and security. Finally, in Section 13 we conclude pressed raw video streams require excessivethe paper. bandwidth for a multi-hop wireless environment. For example, a single monochrome frame in2. Factors inﬂuencing the design of multimedia sensor the NTSC-based Quarter Common Intermediatenetworks Format (QCIF, 176 · 120), requires around 21 Kbyte, and at 30 frames per second (fps), a Wireless Multimedia Sensor Networks (WMSNs) video stream requires over 5 Mbit/s. Hence, it iswill be enabled by the convergence of communica- apparent that eﬃcient processing techniques for
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 925 lossy compression are necessary for multimedia source, i.e., each application processes its desired sensor networks. Traditional video coding tech- sensor feeds on the CPU of the sensor nodes niques used for wireline and wireless communica- where data are gathered. This dramatically tions are based on the idea of reducing the bit reduces the bandwidth consumed, since instead rate generated by the source encoder by exploit- of transferring raw data, IrisNet sends only a ing source statistics. To this aim, encoders rely potentially small amount of processed data. on intra-frame compression techniques to reduce However, the cost of multimedia processing algo- redundancy within one frame, while they leverage rithms may be prohibitive for low-end multime- inter-frame compression (also known as predic- dia sensors. Hence, it is necessary to develop tive encoding or motion estimation) to exploit scalable and energy-eﬃcient distributed ﬁltering redundancy among subsequent frames to reduce architectures to enable processing of redundant the amount of data to be transmitted and stored, data as close as possible to the periphery of the thus achieving good rate-distortion performance. network. Since predictive encoding requires complex • Power consumption. Power consumption is a fun- encoders, powerful processing algorithms, and damental concern in WMSNs, even more than in entails high energy consumption, it may not be traditional wireless sensor networks. In fact, sen- suited for low-cost multimedia sensors. However, sors are battery-constrained devices, while multi- it has recently been shown  that the tradi- media applications produce high volumes of tional balance of complex encoder and simple data, which require high transmission rates, and decoder can be reversed within the framework extensive processing. While the energy consump- of the so-called distributed source coding, which tion of traditional sensor nodes is known to be exploits the source statistics at the decoder, and dominated by the communication functionalities, by shifting the complexity at this end, allows this may not necessarily be true in WMSNs. the use of simple encoders. Clearly, such algo- Therefore, protocols, algorithms and architec- rithms are very promising for WMSNs and espe- tures to maximize the network lifetime while pro- cially for networks of video sensors, where it may viding the QoS required by the application are a not be feasible to use existing video encoders at critical issue. the source node due to processing and energy • Flexible architecture to support heterogeneous constraints. applications. WMSN architectures will support• Multimedia in-network processing. WMSNs allow several heterogeneous and independent applica- performing multimedia in-network processing tions with diﬀerent requirements. It is necessary algorithms on the raw data extracted from the to develop ﬂexible, hierarchical architectures that environment. This requires new architectures can accommodate the requirements of all these for collaborative, distributed, and resource-con- applications in the same infrastructure. strained processing that allow for ﬁltering and • Multimedia coverage. Some multimedia sensors, extraction of semantically relevant information in particular video sensors, have larger sensing at the edge of the sensor network. This may radii and are sensitive to direction of acquisition increase the system scalability by reducing the (directivity). Furthermore, video sensors can cap- transmission of redundant information, merging ture images only when there is unobstructed line data originated from multiple views, on diﬀerent of sight between the event and the sensor. Hence, media, and with multiple resolutions. For exam- coverage models developed for traditional wire- ple, in video security applications, information less sensor networks are not suﬃcient for pre- from uninteresting scenes can be compressed to deployment planning of a multimedia sensor a simple scalar value or not be transmitted network. altogether, while in environmental applications, • Integration with Internet (IP) architecture. It is of distributed ﬁltering techniques can create a fundamental importance for the commercial time-elapsed image . Hence, it is necessary development of sensor networks to provide ser- to develop application-independent architectures vices that allow querying the network to retrieve to ﬂexibly perform in-network processing of the useful information from anywhere and at any multimedia content gathered from the environ- time. For this reason, future WMSNs will ment. For example, IrisNet  uses applica- be remotely accessible from the Internet, and tion-speciﬁc ﬁltering of sensor feeds at the will therefore need to be integrated with the IP
926 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 architecture. The characteristics of WSNs rule the growing size of the network. Flat topologies out the possibility of all-IP sensor networks and may not always be suited to handle the amount of recommend the use of application level gateways traﬃc generated by multimedia applications includ- or overlay IP networks as the best approach for ing audio and video. Likewise, the processing power integration between WSNs and the Internet required for data processing and communications, . and the power required to operate it, may not be• Integration with other wireless technologies. available on each node. Large-scale sensor networks may be created by interconnecting local ‘‘islands’’ of sensors through other wireless technologies. This needs 3.1. Reference architecture to be achieved without sacriﬁcing on the eﬃ- ciency of the operation within each individual In Fig. 1, we introduce a reference architecture technology. for WMSNs, where three sensor networks with dif- ferent characteristics are shown, possibly deployed in diﬀerent physical locations. The ﬁrst cloud on the left shows a single-tier network of homogeneous3. Network architecture video sensors. A subset of the deployed sensors have higher processing capabilities, and are thus referred The problem of designing a scalable network to as processing hubs. The union of the processingarchitecture is of primary importance. Most propos- hubs constitutes a distributed processing architec-als for wireless sensor networks are based on a ﬂat, ture. The multimedia content gathered is relayedhomogenous architecture in which every sensor has to a wireless gateway through a multi-hop path.the same physical capabilities and can only interact The gateway is interconnected to a storage hub, thatwith neighboring sensors. Traditionally, the research is in charge of storing multimedia content locallyon algorithms and protocols for sensor networks for subsequent retrieval. Clearly, more complexhas focused on scalability, i.e., how to design solu- architectures for distributed storage can be imple-tions whose applicability would not be limited by mented when allowed by the environment and the Fig. 1. Reference architecture of a wireless multimedia sensor network.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 927application needs, which may result in energy sav- of scalability, lower cost, better coverage, higherings since by storing it locally, the multimedia functionality, and better reliability.content does not need to be wirelessly relayed toremote locations. The wireless gateway is also 3.3. Coverageconnected to a central sink, which implements thesoftware front-end for network querying and In traditional WSNs, sensor nodes collect infor-tasking. The second cloud represents a single-tiered mation from the environment within a pre-deﬁnedclustered architecture of heterogeneous sensors sensing range, i.e., a roughly circular area deﬁned(only one cluster is depicted). Video, audio, and by the type of sensor being used.scalar sensors relay data to a central clusterhead, Multimedia sensors generally have larger sensingwhich is also in charge of performing intensive mul- radii and are also sensitive to the direction of datatimedia processing on the data (processing hub). acquisition. In particular, cameras can captureThe clusterhead relays the gathered content to the images of objects or parts of regions that are notwireless gateway and to the storage hub. The last necessarily close to the camera itself. However, thecloud on the right represents a multi-tiered network, image can obviously be captured only when therewith heterogeneous sensors. Each tier is in charge is an unobstructed line-of-sight between the eventof a subset of the functionalities. Resource-con- and the sensor. Furthermore, each multimediastrained, low-power scalar sensors are in charge sensor/camera perceives the environment or theof performing simpler tasks, such as detecting observed object from a diﬀerent and unique view-scalar physical measurements, while resource-rich, point, given the diﬀerent orientations and positionshigh-power devices are responsible for more com- of the cameras relative to the observed event orplex tasks. Data processing and storage can be region. In , a preliminary investigation of theperformed in a distributed fashion at each diﬀerent coverage problem for video sensor networks is con-tier. ducted. The concept of sensing range is replaced with the camera’s ﬁeld of view, i.e., the maximum3.2. Single-tier vs. multi-tier sensor deployment volume visible from the camera. It is also shown how an algorithm designed for traditional sensor One possible approach for designing a multime- networks does not perform well with video sensorsdia sensor application is to deploy homogeneous in terms of coverage preservation of the monitoredsensors and program each sensor to perform all pos- area.sible application tasks. Such an approach yields aﬂat, single-tier network of homogeneous sensor 4. Multimedia sensor hardwarenodes. An alternative, multi-tier approach is to useheterogeneous elements . In this approach, In this section, we review and classify existingresource-constrained, low-power elements are in imaging, multimedia, and processing wirelesscharge of performing simpler tasks, such as detect- devices that will ﬁnd application in next generationing scalar physical measurements, while resource- wireless multimedia sensor networks. In particular,rich, high-power devices take on more complex we discuss existing hardware, with a particulartasks. For instance, a surveillance application can emphasis on video capturing devices, review existingrely on low-ﬁdelity cameras or scalar acoustic sen- implementations of multimedia sensor networks,sors to perform motion or intrusion detection, while and discuss current possibilities for energy harvest-high-ﬁdelity cameras can be woken up on-demand ing for multimedia sensor devices.for object recognition and tracking. In , amulti-tier architecture is advocated for video sensor 4.1. Enabling hardware platformsnetworks for surveillance applications. The architec-ture is based on multiple tiers of cameras with diﬀer- High-end pan-tilt-zoom cameras and high resolu-ent functionalities, with the lower tier constituted of tion digital cameras are widely available on the mar-low-resolution imaging sensors, and the higher tier ket. However, while such sophisticated devices cancomposed of high-end pan-tilt-zoom cameras. It is ﬁnd application as high-quality tiers of multimediaargued, and shown by means of experiments, that sensor networks, we concentrate on low-cost, low-such an architecture oﬀers considerable advantages energy consumption imaging and processing deviceswith respect to a single-tier architecture in terms that will be densely deployed and provide detailed
928 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960visual information from multiple disparate view- Researchers at Carnegie Mellon University arepoints, help overcoming occlusion eﬀects, and thus developing the CMUcam 3, which is an embeddedenable enhanced interaction with the environment. camera endowed with a CIF Resolution (352 · 288) RGB color sensor that can load images into memory4.1.1. Low-resolution imaging motes at 26 frames per second. CMUcam 3 has software The recent availability of CMOS imaging sensors JPEG compression and has a basic image manipula- that capture and process an optical image within tion library, and can be interface with an 802.15.4a single integrated chip, thus eliminating the need for compliant TelosB mote .many separate chips required by the traditional In , the design of an integrated mote for wire-charged-coupled device (CCD) technology, has less image sensor networks is described. The designenabled the massive deployment of low-cost visual is driven by the need to endow motes with adequatesensors. CMOS image sensors are already in many processing power and memory size for image sens-industrial and consumer sectors, such as cell phones, ing applications. It is argued that 32-bit processorspersonal digital assistants (PDAs), consumer and are better suited for image processing than their 8-industrial digital cameras. CMOS image quality is bit counterpart, which is used in most existingnow matching CCD quality in the low- and mid- motes. It is shown that the time needed to performrange, while CCD is still the technology of choice operations such as 2-D convolution on an 8-bit pro-for high-end image sensors. The CMOS technology cessor such as the ATMEL ATmega128 clocked atallows integrating a lens, an image sensor and image 4 MHz is 16 times higher than with a 32-bitprocessing algorithms, including image stabilization ARM7 device clocked at 48 MHz, while the powerand image compression, on the same chip. With consumption of the 32-bit processor is only six timesrespect to CCD, cameras are smaller, lighter, and higher. Hence, an 8-bit processor turns out to beconsume less power. Hence, they constitute a suit- slower and more energy-consuming. Based on theseable technology to realize imaging sensors to be premises, a new image mote is developed based oninterfaced with wireless motes. an ARM7 32-bit CPU clocked at 48 MHz, with However, existing CMOS imagers are still external FRAM or Flash memory, 802.15.4 compli-designed to be interfaced with computationally rich ant Chipcon CC2420 radio, that is interfaced withhost devices, such as cell phones or PDAs. For this mid-resolution ADCM-1670 CIF CMOS sensorsreason, the objective of the Cyclops module  is and low-resolution 30 · 30 pixel optical sensors.to ﬁll the gap between CMOS cameras and compu- The same conclusion is drawn in , where thetationally constrained devices. Cyclops is an elec- energy consumption of the 8-bit Atmel AVR pro-tronic interface between a CMOS camera module cessor clocked at 8 MHz is compared to that ofand a wireless mote such as MICA2 or MICAz, the PXA255 32-bit Intel processor, embedded on aand contains programmable logic and memory for Stargate platform  and clocked at 400 MHz.high-speed data communication. Cyclops consists Three representative algorithms are selected asof an imager (CMOS Agilent ADCM-1700 CIF benchmarks, i.e., the cyclic redundancy check, acamera), an 8-bit ATMEL ATmega128L microcon- ﬁnite impulse response ﬁlter, and a fast Fouriertroller (MCU), a complex programmable logic transform. Surprisingly, it is shown that even fordevice (CPLD), an external SRAM and an external such relatively simple algorithms the energy con-Flash. The MCU controls the imager, conﬁgures its sumption of an 8-bit processor is between one andparameters, and performs local processing on the two orders of magnitude higher.image to produce an inference. Since image capturerequires faster data transfer and address generation 4.1.2. Medium-resolution imaging motes based on thethan the 4 MHz MCU used, a CPLD is used to pro- Stargate platformvide access to the high-speed clock. Cyclops ﬁrm- Intel has developed several prototypes that con-ware is written in the nesC language , based on stitute important building platform for WMSNthe TinyOS libraries. The module is connected to applications. The Stargate board  is a high-per-a host mote to which it provides a high level inter- formance processing platform designed for sensor,face that hides the complexity of the imaging device signal processing, control, robotics, and sensor net-to the host mote. Moreover, it can perform simple work applications. It is designed by Intel and pro-inference on the image data and present it to the duced by Crossbow. Stargate is based on Intel’shost. PXA-255 XScale 400 MHz RISC processor, which
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 929is the same processor found in many handheld com- Systems able to perpetually power sensors basedputers including the Compaq IPAQ and the Dell on simple COTS photovoltaic cells coupled withAxim. Stargate has 32 Mbyte of Flash memory, supercapacitors and rechargeable batteries have64 Mbyte of SDRAM, and an on-board connector been already demonstrated . In , the statefor Crossbow’s MICA2 or MICAz motes as well of the art in more unconventional techniques foras PCMCIA Bluetooth or IEEE 802.11 cards. energy harvesting (also referred to as energy scav-Hence, it can work as a wireless gateway and as a enging) is surveyed. Technologies to generate energycomputational hub for in-network processing algo- from background radio signals, thermoelectric con-rithms. When connected with a webcam or other version, vibrational excitation, and the humancapturing device, it can function as a medium-reso- body, are overviewed.lution multimedia sensor, although its energy con- As far as collecting energy from backgroundsumption is still high, as documented in . radio signals is concerned, unfortunately, an electricMoreover, although eﬃcient software implementa- ﬁeld of 1 V/m yields only 0.26 lW/cm2, as opposedtions exist, XScale processors do not have hardware to 100 lW/cm2 produced by a crystalline siliconsupport for ﬂoating point operations, which may be solar cell exposed to bright sunlight. Electric ﬁeldsneeded to eﬃciently perform multimedia processing of intensity of a few volts per meter are only encoun-algorithms. tered close to strong transmitters. Another practice, Intel has also developed two prototypal genera- which consists in broadcasting RF energy deliber-tions of wireless sensors, known as Imote and ately to power electronic devices, is severely limitedImote2. Imote is built around an integrated wireless by legal limits set by health and safety concerns.microcontroller consisting of an 8-bit 12 MHz While thermoelectric conversion may not be suit-ARM7 processor, a Bluetooth radio, 64 Kbyte able for wireless devices, harvesting energy fromRAM and 32 Kbyte FLASH memory, as well as vibrations in the surrounding environment may pro-several I/O options. The software architecture is vide another useful source of energy. Vibrationalbased on an ARM port of TinyOS. The second gen- magnetic power generators based on moving mag-eration of Intel motes has a common core to the nets or coils may yield powers that range from tensnext generation Stargate 2 platform, and is built of microwatts when based on microelectromechani-around a new low-power 32-bit PXA271 XScale cal system (MEMS) technologies to over a milliwattprocessor at 320/416/520 MHz, which enables per- for larger devices. Other vibrational microgenera-forming DSP operations for storage or compres- tors are based on charged capacitors with movingsion, and an IEEE 802.15.4 ChipCon CC2420 plates, and depending on their excitation and powerradio. It has large on-board RAM and Flash mem- conditioning, yield power on the order of 10 lW. Inories (32 Mbyte), additional support for alternate , it is also reported that recent analysis  sug-radios, and a variety of high-speed I/O to connect gested that 1 cm3 vibrational microgenerators candigital sensors or cameras. Its size is also very lim- be expected to yield up to 800 lW/cm3 fromited, 48 · 33 mm, and it can run the Linux operating machine-induced stimuli, which is orders of magni-system and Java applications. tude higher than what provided by currently avail- able microgenerators. Hence, this is a promising4.2. Energy harvesting area of research for small battery-powered devices. While these techniques may provide an addi- As mentioned before, techniques for prolonging tional source of energy and help prolong the lifetimethe lifetime of battery-powered sensors have been of sensor devices, they yield power that is severalthe focus of a vast amount of literature in sensor orders of magnitude lower as compared to thenetworks. These techniques include hardware opti- power consumption of state-of-the-art multimediamizations such as dynamic optimization of voltage devices. Hence, they may currently be suitable onlyand clock rate, wake-up procedures to keep elec- for very-low duty cycle devices.tronics inactive most of the time, and energy-awareprotocol development for sensor communications. 4.3. Examples of deployed multimedia sensorIn addition, energy-harvesting techniques, which networksextract energy from the environment where the sen-sor itself lies, oﬀer another important mean to pro- There have been several recent experimentallong the lifetime of sensor devices. studies, mostly limited to video sensor networks.
930 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960Panoptes  is a system developed for environmen- ﬁdelity Cyclops  or CMUcam  camera sen-tal observation and surveillance applications, based sors. The third tier consists of Stargate  nodeson Intel StrongARM PDA platforms with a Logi- equipped with webcams. Each Stargate is equippedtech webcam as a video capture device. Here, video with an embedded 400 MHz XScale processor thatsensors are high-end devices with Linux operating runs Linux and a webcam that can capture highersystem, 64 Mbyte of memory, and are networked ﬁdelity images than tier 2 cameras. Tier 3 nodes alsothrough 802.11 networking cards. The system perform gateway functions, as they are endowedincludes spatial compression (but not temporal), with a low data rate radio to communicate withdistributed ﬁltering, buﬀering, and adaptive priori- motes in tiers 1–2 at 900 MHz, and an 802.11 radioties for the video stream. to communicate with tier 3 Stargate nodes. An addi- In , a system whose objective is to limit the tional fourth tier may consist of a sparse deploy-computation, bandwidth, and human attention bur- ment of high-resolution, high-end pan-tilt-zoomdens imposed by large-scale video surveillance sys- cameras connected to embedded PCs. The cameratems is described. In-network processing is used sensors at this tier can be used to track movingon each camera to ﬁlter out uninteresting events objects, and can be utilized to ﬁll coverage gapslocally, avoiding disambiguation and tracking of and provide additional redundancy. The underlyingirrelevant environmental distractors. A resource design principle is to map each task requested by theallocation algorithm is also proposed to steer pan- application to the lowest tier with suﬃcienttilt cameras to follow interesting targets while main- resources to perform the task. Devices from highertaining awareness of possibly emerging new targets. tiers are woken up on-demand only when necessary. In , the design and implementation of Sens- For example, a high-resolution camera can beEye, a multi-tier network of heterogeneous wireless woken up to retrieve high resolution images of annodes and cameras, is described. The surveillance object that has been previously detected by a lowerapplication consists of three tasks: object detection, tier. It is shown that the system can achieve an orderrecognition and tracking. The objective of the of magnitude reduction in energy consumptiondesign is to demonstrate that a camera sensor net- while providing comparable surveillance accuracywork containing heterogeneous elements provides with respect to single-tier surveillance systems.numerous beneﬁts over traditional homogeneous In , experimental results on the energy con-sensor networks. For this reason, SensEye follows sumption of a video sensor network testbed are pre-a three-tier architecture, as shown in Fig. 2. The sented. Each sensing node in the testbed consists oflowest tier consists of low-end devices, i.e., MICA2 a Stargate board equipped with an 802.11 wirelessMotes equipped with 900 MHz radios interfaced network card and a Logitech QuickCam Pro 4000with scalar sensors, e.g., vibration sensors. The sec- webcam. The energy consumption is assessed usingond tier is made up of motes equipped with low- a benchmark that runs basic tasks such as process- ing, ﬂash memory access, image acquisition, and communication over the network. Both steady state Video stream and transient energy consumption behavior Tier 3 handoff obtained by direct measurements of current with a digital multimeter are reported. In the steady state, it is shown that communication-related tasks are Webcam + Stargate wakeup less energy-consuming than intensive processing and ﬂash access when the radio modules are loaded. Tier 2 Interestingly, and unlike in traditional wireless sen- sor networks , the processing-intensive bench- Low-res cam + Mote mark results in the highest current requirement, wakeup and transmission is shown to be only about 5% more energy-consuming than reception. Experimen- Tier 1 tal results also show that delay and additional amount of energy consumed due to transitions Scalar Sensors + (e.g., to go to sleep mode) are not negligible and Mote must be accounted for in network and protocol Fig. 2. The multi-tier architecture of SensEye . design.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 931 IrisNet (Internet-scale Resource-Intensive Sensor allows testing more complex algorithms and assessNetwork Services)  is an example software plat- the scalability of the communication protocolsform to deploy heterogeneous services on WMSNs. under examination.IrisNet allows harnessing a global, wide-area sensor The WMSN-testbed includes three diﬀerent typesnetwork by performing Internet-like queries on this of multimedia sensors: low-end imaging sensors,infrastructure. Video sensors and scalar sensors are medium-quality webcam-based multimedia sensors,spread throughout the environment, and collect and pan-tilt cameras mounted on mobile robots.potentially useful data. IrisNet allows users to per- Low-end imaging sensors such as CMOS cam-form Internet-like queries to video sensors and eras can be interfaced with Crossbow MICAzother data. The user views the sensor network as a motes. Medium-end video sensors are based onsingle unit that can be queried through a high-level Logitech webcams interfaced with Stargate plat-language. Each query operates over data collected forms (see Fig. 3).from the global sensor network, and allows simple The high-end video sensors consist of pan-tiltGoogle-like queries as well as more complex queries cameras installed on an Acroname GARCIAinvolving arithmetic and database operators. The architecture of IrisNet is two-tiered: hetero-geneous sensors implement a common shared inter-face and are called sensing agents (SA), while thedata produced by sensors is stored in a distributeddatabase that is implemented on organizing agents(OA). Diﬀerent sensing services are run simulta-neously on the architecture. Hence, the same hard-ware infrastructure can provide diﬀerent sensingservices. For example, a set of video sensors canprovide a parking space ﬁnder service, as well as asurveillance service. Sensor data is represented inthe Extensible Markup Language (XML), whichallows easy organization of hierarchical data. Agroup of OAs is responsible for a sensing service,collects data produced by that service, and orga- Fig. 3. Stargate board interfaced with a medium resolutionnizes the information in a distributed database to camera. Stargate hosts an 802.11 card and a MICAz mote thatanswer the class of relevant queries. IrisNet also functions as a gateway to the sensor network.allows programming sensors with ﬁltering code thatprocesses sensor readings in a service-speciﬁc way.A single SA can execute several such software ﬁlters(called senselets) that process the raw sensor databased on the requirements of the service that needsto access the data. After senselet processing, the dis-tilled information is sent to a nearby OA. We have recently built an experimental testbedat the Broadband and Wireless Networking(BWN) Laboratory at Georgia Tech based on cur-rently oﬀ-the-shelf advanced devices to demonstratethe eﬃciency of algorithms and protocols for multi-media communications through wireless sensornetworks. The testbed is integrated with our scalar sensornetwork testbed, which is composed of a heteroge-neous collection of imotes from Intel and MICAzmotes from Crossbow. Although our testbed Fig. 4. Acroname GARCIA, a mobile robot with a mountedalready includes 60 scalar sensors, we plan to pan-tilt camera and endowed with 802.11 as well as Zigbeeincrease its size to deploy a higher scale testbed that interfaces.
932 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 processing architectures, to allow real-time retrieval of useful information. Similarly, it is necessary to develop architectures that eﬃciently allow performing data fusion or other complex processing operations in-network. Algorithms for both inter-media and intra-media data aggregation and fusion need to be developed, as simple distributed processing schemes developed for existing scalar sensors are not suitable for com- putation-intensive processing required by multime- dia contents. Multimedia sensor networks may require computation-intensive processing algo- rithms (e.g., to detect the presence of suspiciousFig. 5. GARCIA deployed on the sensor testbed. It acts as a activity from a video stream). This may require con-mobile sink, and can move to the area of interest for closer visual siderable processing to extract meaningful informa-inspection. It can also coordinate with other actors and has built- tion and/or to perform compression. A fundamentalin collision avoidance capability. question to be answered is whether this processing can be done on sensor nodes (i.e., a ﬂat architecture of multi-functional sensors that can perform anyrobotic platform , which we refer to as actor, and task), or if the need for specialized devices, e.g.,shown in Fig. 4. Actors constitute a mobile platform computation hubs, arises.that can perform adaptive sampling based on event In what follows, we discuss a non-exhaustive setfeatures detected by low-end motes. The mobile of signiﬁcative examples of processing techniquesactor can redirect high-resolution cameras to a that would be applicable distributively in a WMSN,region of interest when events are detected by and that will likely drive research on architectureslower-tier, low-resolution video sensors that are and algorithms for distributed processing of rawdensely deployed, as seen in Fig. 5. sensor data. The testbed also includes storage and computa-tional hubs, which are needed to store large multi- 5.1. Data alignment and image registrationmedia content and perform computationallyintensive multimedia processing algorithms. Data alignment consists of merging information from multiple sources. One of the most widespread5. Collaborative in-network processing data alignment concepts, image registration , is a family of techniques, widely used in areas such as As discussed previously, collaborative in-net- remote sensing, medical imaging, and computerwork multimedia processing techniques are of great vision, to geometrically align diﬀerent images (refer-interest in the context of a WMSN. It is necessary to ence and sensed images) of the same scene taken atdevelop architectures and algorithms to ﬂexibly per- diﬀerent times, from diﬀerent viewpoints, and/or byform these functionalities in-network with minimum diﬀerent sensors:energy consumption and limited execution time.The objective is usually to avoid transmitting large • Diﬀerent Viewpoints (Multi-view Analysis). Imagesamounts of raw streams to the sink by processing of the same scene are acquired from diﬀerentthe data in the network to reduce the communica- viewpoints, to gain a larger 2D view or a 3D rep-tion volume. resentation of the scene of interest. Main applica- Given a source of data (e.g., a video stream), dif- tions are in remote sensing, computer vision andferent applications may require diverse information 3D shape recovery.(e.g., raw video stream vs. simple scalar or binary • Diﬀerent times (multi-temporal analysis). Imagesinformation inferred by processing the video of the same scene are acquired at diﬀerent times.stream). This is referred to as application-speciﬁc The aim is to ﬁnd and evaluate changes in time inquerying and processing. Hence, it is necessary to the scene of interest. The main applications aredevelop expressive and eﬃcient querying languages, in computer vision, security monitoring, andand to develop distributed ﬁltering and in-network motion tracking.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 933• Diﬀerent sensors (multi-modal analysis). Images coordinate computations across the vision nodes of the same scene are acquired by diﬀerent sen- and return the integrated results, which will consist sors. The objective is to integrate the information of metadata information, to the ﬁnal user. obtained from diﬀerent source streams to gain In , the proposed Deep Vision network per- more complex and detailed scene representation. forms operations including object detection or clas- siﬁcation, image segmentation, and motion analysis Registration methods usually consist of four through a network of low-end MICA motessteps, i.e., feature detection, feature matching, trans- equipped with Cyclops cameras . Informationform model estimation, and image resampling and such as the presence of an intruder, the number oftransformation. In feature detection, distinctive visitors in a scene or the probability of presence ofobjects such as closed-boundary regions, edges, con- a human in the monitored area is obtained by col-tours, line intersections, corners, etc. are detected. lecting the results of these operations. Deep VisionIn feature matching, the correspondence between provides a querying interface to the user in the formthe features detected in the sensed image and those of declarative queries. Each operation is representeddetected in the reference image is established. In as an attribute that can be executed through antransform model estimation, the type and parame- appropriate query. In this way, low-level operationsters of the so-called mapping functions, which align and processing are encapsulated in a high-level que-the sensed image with the reference image, are esti- rying interface that enables simple interaction withmated. The parameters of the mapping functions the video network. As an example, the vision net-are computed by means of the established feature work can be deployed in areas with public andcorrespondence. In the last step, image resampling restricted access spaces. The task of detectingand transformation, the sensed image is trans- objects in the restricted-access area can be expressedformed by means of the mapping functions. as a query that requests the result of object detec- These functionalities can clearly be prohibitive tion computations such asfor a single sensor. Hence, research is needed onhow to perform these functionalities on parallel SELECT Object,Locationarchitectures of sensors to produce single data sets. REPORT = 30 FROM Network5.2. WMSNs as distributed computer vision systems WHERE Access = Restricted PERIOD = 30. Computer vision is a subﬁeld of artiﬁcial intelli-gence, whose purpose is to allow a computer to The above query triggers the execution of theextract features from a scene, an image or multi- object detection process on the vision nodes thatdimensional data in general. The objective is to are located in the restricted-access areas in 30 spresent this information to a human operator or intervals.to control some process (e.g., a mobile robot or anautonomous vehicle). The image data that is fed 6. Application layerinto a computer vision system is often a digitalimage, a video sequence, a 3D volume from a The functionalities handled at the applicationtomography device or other multimedia content. layer of a WMSN are characterized by high hetero-Traditional computer vision algorithms require geneity, and encompass traditional communicationextensive computation, which in turn entails high problems as well as more general system challenges.power consumption. The services oﬀered by the application layer include: WMSNs enable a new approach to computer (i) providing traﬃc management and admission con-vision, where visual observations across the network trol functionalities, i.e., prevent applications fromcan be performed by means of distributed computa- establishing data ﬂows when the network resourcestions on multiple, possibly low-end, vision nodes. needed are not available; (ii) performing sourceThis requires tools to interface with the user such coding according to application requirements andas new querying languages and abstractions to hardware constraints, by leveraging advanced mul-express complex tasks that are then distributively timedia encoding techniques; (iii) providing ﬂexibleaccomplished through low-level operations on mul- and eﬃcient system software, i.e., operating systemstiple vision nodes. To this aim, it is necessary to and middleware, to export services for higher-layer
934 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960applications to build upon; (iv) providing primitives but the application is moderately loss-tolerant.for applications to leverage collaborative, advanced The bandwidth demand is usually between lowin-network multimedia processing techniques. In and moderate.this section, we provide an overview of these • Real-time, Loss-intolerant, Data. This may includechallenges. data from time-critical monitoring processes such as distributed control applications. The band-6.1. Traﬃc classes width demand varies between low and moderate. • Delay-tolerant, Loss-intolerant, Data. This may Admission control has to be based on QoS include data from critical monitoring processes,requirements of the overlying application. We envi- with low or moderate bandwidth demand thatsion that WMSNs will need to provide support and require some form of oﬄine post processing.diﬀerentiated service for several diﬀerent classes • Delay-tolerant, Loss-tolerant, Data. This mayof applications. In particular, they will need to include environmental data from scalar sensorprovide diﬀerentiated service between real-time networks, or non-time-critical snapshot multime-and delay-tolerant applications, and loss-tolerant dia content, with low or moderate bandwidthand loss-intolerant applications. Moreover, some demand.applications may require a continuous stream ofmultimedia data for a prolonged period of time QoS requirements have recently been considered(multimedia streaming), while some other applica- as application admission criteria for sensor networks.tions may require event triggered observations In , an application admission control algorithm isobtained in a short time period (snapshot multimedia proposed whose objective is to maximize the networkcontent). The main traﬃc classes that need to be lifetime subject to bandwidth and reliability con-supported are: straints of the application. An application admission control method is proposed in , which determines• Real-time, Loss-tolerant, Multimedia Streams. admissions based on the added energy load and This class includes video and audio streams, or application rewards. While these approaches address multi-level streams composed of video/audio application level QoS considerations, they fail to con- and other scalar data (e.g., temperature read- sider multiple QoS requirements (e.g., delay, reliabil- ings), as well as metadata associated with the ity, and energy consumption) simultaneously, as stream, that need to reach a human or automated required in WMSNs. Furthermore, these solutions operator in real-time, i.e., within strict delay do not consider the peculiarities of WMSNs, i.e., they bounds, and that are however relatively loss tol- do not try to base admission control on a tight bal- erant (e.g., video streams can be within a certain ancing between communication optimizations and level of distortion). Traﬃc in this class usually in-network computation. There is a clear need for has high bandwidth demand. new criteria and mechanisms to manage the admis-• Delay-tolerant, Loss-tolerant, Multimedia Streams. sion of multimedia ﬂows according to the desired This class includes multimedia streams that, being application-layer QoS. intended for storage or subsequent oﬄine process- ing, do not need to be delivered within strict delay 6.2. Multimedia encoding techniques bounds. However, due to the typically high band- width demand of multimedia streams and to lim- There exists a vast literature on multimedia ited buﬀers of multimedia sensors, data in this encoding techniques. The captured multimedia con- traﬃc class needs to be transmitted almost in tent should ideally be represented in such a way as real-time to avoid excessive losses. to allow reliable transmission over lossy channels• Real-time, Loss-tolerant, Data. This class may (error-resilient coding), using algorithms that mini- include monitoring data from densely deployed mize processing power and the amount of informa- scalar sensors such as light sensors whose moni- tion to be transmitted. The main design objectives tored phenomenon is characterized by spatial of a coder for multimedia sensor networks are thus: correlation, or loss-tolerant snapshot multimedia data (e.g., images of a phenomenon taken from • High compression eﬃciency. Uncompressed raw several multiple viewpoints at the same time). video streams require high data rates and thus Hence, sensor data has to be received timely consume excessive bandwidth and energy. It is
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 935 necessary to achieve a high ratio of compres- refers to the compression of multiple correlated sen- sion to eﬀectively limit bandwidth and energy sor outputs that do not communicate with each consumption. other . Joint decoding is performed by a central• Low complexity. Multimedia encoders are entity that receives data independently compressed embedded in sensor devices. Hence, they need by diﬀerent sensors. However, practical solutions to be low complexity to reduce cost and form fac- have not been developed until recently. Clearly, tors, and low-power to prolong the lifetime of such techniques are very promising for WMSNs sensor nodes. and especially for networks of video sensors. The• Error resiliency. The source coder should provide encoder can be simple and low-power, while the robust and error-resilient coding of source data. decoder at the sink will be complex and loaded with most of the processing and energy burden. The To achieve a high compression eﬃciency, the tra- reader is referred to [131,50] for excellent surveysditional broadcasting paradigm for wireline and on the state of the art of distributed source codingwireless communications, where video is com- in sensor networks and in distributed video coding,pressed once at the encoder and decoded several respectively. Other encoding and compressiontimes, has been dominated by predictive encoding schemes that may be considered for source codingtechniques. These, used in the widely spread ISO of multimedia streams, including JPEG with diﬀer-MPEG schemes, or the ITU-T recommendations ential encoding, distributed coding of images takenH.263  and H.264  (also known as AVC or by cameras having overlapping ﬁelds of view, orMPEG-4 part 10), are based on the idea of reducing multi-layer coding with wavelet compression, arethe bit rate generated by the source encoder by discussed in . Here, we focus on recent advancesexploiting source statistics. Hence, intra-frame com- on low complexity encoders based on Wyner–Zivpression techniques are used to reduce redundancy coding , which are promising solutions for dis-within one frame, while inter-frame compression tributed networks of video sensors that are likely to(also known as predictive encoding or motion estima- have a major impact in future design of protocolstion) exploits correlation among subsequent frames for WMSNs.to reduce the amount of data to be transmitted The objective of a Wyner–Ziv video coder is toand stored, thus achieving good rate-distortion per- achieve lossy compression of video streams andformance. Since the computational complexity is achieve performance comparable to that of inter-dominated by the motion estimation functionality, frame encoding (e.g., MPEG), with complexity atthese techniques require complex encoders, power- the encoder comparable to that of intra-frame cod-ful processing algorithms, and entail high energy ers (e.g., Motion-JPEG).consumption, while decoders are simpler and loadedwith lower processing burden. For typical imple- 6.2.1. Pixel-domain Wyner–Ziv encodermentations of state-of-the-art video compression In [14,15], a practical Wyner–Ziv encoder is pro-standards, such as MPEG or H.263 and H.264, posed as a combination of a pixel-domain intra-the encoder is 5–10 times more complex than the frame encoder and inter-frame decoder system fordecoder . It is easy to see that to realize low-cost, video compression. A block diagram of the systemlow-energy-consumption multimedia sensors it is is reported in Fig. 6. A regularly spaced subset ofnecessary to develop simpler encoders, and still frames is coded using a conventional intra-frameretain the advantages of high compression coding technique, such as JPEG, as shown at theeﬃciency. bottom of the ﬁgure. These are referred to as key However, it is known from information-theoretic frames. All frames between the key frames arebounds established by Slepian and Wolf for lossless referred to as Wyner–Ziv frames and are intra-framecoding  and by Wyner and Ziv  for lossy encoded but inter-frame decoded. The intra-framecoding with decoder side information, that eﬃcient encoder for Wyner–Ziv frames (shown on top) iscompression can be achieved by leveraging knowl- composed of a quantizer followed by a Slepian–edge of the source statistics at the decoder only. This Wolf coder. Each Wyner–Ziv frame is quantizedway, the traditional balance of complex encoder and and blocks of symbols are sent to the Slepian–Wolfsimple decoder can be reversed . Techniques that coder, which is implemented through rate-compati-build upon these results are usually referred to as ble punctured turbo codes (RCPT). The parity bitsdistributed source coding. Distributed source coding generated by the RCPT coder are stored in a buﬀer.
936 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 INTRAFRAME ENCODER INTERFRAME DECODER WYNER-ZIV SLEPIAN-WOLF CODER FRAMES RCPT RCPT QUANTIZER BUFFER RECONSTRUCTION ENCODER DECODER DECODED WYNER-ZIV FRAMES SIDE INFORMATION DECODER FEEDBACK E REQUEST ADDITIONAL BITS INTERPOLATION AND EXTRAPOLATION DECODED KEY INTRAFRAME KEY FRAMES FRAMES ENCODER INTRAFRAME (E.G. JPEG) DECODER Fig. 6. Block diagram of a pixel-domain Wyner–Ziv encoder .A subset of these bits is then transmitted upon 6.2.2. Transform-domain Wyner–Ziv encoderrequest from the decoder. This allows adapting the In conventional source coding, a source vector israte based on the temporally varying statistics typically decomposed into spectral coeﬃcients bybetween the Wyner–Ziv frame and the side informa- using orthonormal transforms such as the Discretetion. The parity bits generated by the RCPT coder Cosine Transform (DCT). These coeﬃcients areare in fact used to ‘‘correct’’ the frame interpo- then individually coded with scalar quantizerslated at the decoder. For each Wyner–Ziv frame, and entropy coders. In , a transform-domainthe decoder generates the side information frame Wyner–Ziv encoder is proposed. A block-wiseby interpolation or extrapolation of previously DCT of each Wyner–Ziv frame is performed. Thedecoded key frames and Wyner–Ziv frames. The transform coeﬃcients are independently quantized,side information is leveraged by assuming a Lapla- grouped into coeﬃcient bands, and then com-cian distribution of the diﬀerence between the indi- pressed by a Slepian–Wolf turbo coder. As in thevidual pixels of the original frame and the side pixel-domain encoder described in the previous sec-information. The parameter deﬁning the Laplacian tion, the decoder generates a side information framedistribution is estimated online. The turbo decoder based on previously reconstructed frames. Based oncombines the side information and the parity bits the side information, a bank of turbo decodersto reconstruct the original sequence of symbols. If reconstructs the quantized coeﬃcient bands inde-reliable decoding of the original symbols is impossi- pendently. The rate-distortion performance isble, the turbo decoder requests additional parity bits between conventional intra-frame transform codingfrom the encoder buﬀer. and conventional motion-compensated transform Compared to predictive coding such as MPEG or coding.H.26X, pixel-domain Wyner–Ziv encoding is much A diﬀerent approach consists of allowing somesimpler. The Slepian–Wolf encoder only requires simple temporal dependence estimation at the enco-two feedback shift registers and an interleaver. der to perform rate control without the need forIts performance, in terms of peak signal-to-noise feedback from the receiver. In the PRISM schemeratio (PSNR), is 2–5 dB better than conventional , the encoder selects the coding mode basedmotion-JPEG intra-frame coding. The main draw- on the frame diﬀerence energy between the currentback of this scheme is that it relies on online feed- frame and a previous frame. If the energy of the dif-back from the receiver. Hence it may not be ference is very small, the block is not encoded. If thesuitable for applications where video is encoded block diﬀerence is large, the block is intra-coded.and stored for subsequent use. Moreover, the feed- Between these two situations, one of diﬀerentback may introduce excessive latency for video encoding modes with diﬀerent rates is selected.decoding in a multi-hop network. The rate estimation does not involve motion
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 937compensation and hence is necessarily inaccurate, if not compromising on performance. The principalmotion compensation is used at the decoder. design objective of existing operating systems for sen-Further, the ﬂexibility of the decoder is restricted. sor networks such as TinyOS is high performance. However, their ﬂexibility, inter-operability and rep-6.3. System software and middleware rogrammability are very limited. There is a need for research on systems that allow for this integration. The development of eﬃcient and ﬂexible system We believe that it is of paramount importance tosoftware to make functional abstractions and infor- develop eﬃcient, high level abstractions that willmation gathered by scalar and multimedia sensors enable easy and fast development of sensor networkavailable to higher layer applications is one of the applications. An abstraction similar to the famousmost important challenges faced by researchers to Berkeley TCP sockets, that fostered the develop-manage complexity and heterogeneity of sensor sys- ment of Internet applications, is needed for sensortems. As in , the term system software is used systems. However, diﬀerently from the Berkeleyhere to refer to operating systems, virtual machines, sockets, it is necessary to retain control on the eﬃ-and middleware, which export services to higher- ciency of the low-level operations performed on bat-layer applications. Diﬀerent multimedia sensor net- tery-limited and resource-constrained sensor nodes.work applications are extremely diverse in their As a ﬁrst step towards this direction, Chu et al.requirements and in the way they interact with the  recently proposed Sdlib, a sensor network datacomponents of a sensor system. Hence, the main and communications library built upon the nescdesired characteristics of a system software for language  for applications that require best-WMSNs can be identiﬁed as follows: eﬀort collection of large-size data such as video monitoring applications. The objective of the eﬀort• Provides a high-level interface to specify the is to identify common functionalities shared by behavior of the sensor system. This includes several sensor network applications and to develop semantically rich querying languages that allow a library of thoroughly-tested, reusable and eﬃcient specifying what kind of data is requested from nesC components that abstract high-level opera- the sensor network, the quality of the required tions common to most applications, while leaving data, and how it should be presented to the user; diﬀerences among them to adjustable parameters.• Allows the user to specify application-speciﬁc The library is called Sdlib, Sensor Data Library, algorithms to perform in-network processing on as an analogy to the traditional C++ Standard the multimedia content . For example, the Template Library. Sdlib provides an abstraction user should be able to specify particular image for common operations in sensor networks while processing algorithms or multimedia coding the developer is still able to access low-level opera- format; tions, which are implemented as a collection of nesC• Long-lived, i.e., needs to smoothly support evo- components, when desired. Moreover, to retain eﬃ- lutions of the underlying hardware and software; ciency of operations that are so critical for sensor• Shared among multiple heterogeneous appli- networks battery lifetime and resource constraints, cations; Sdlib exposes policy decisions such as resource allo-• Shared among heterogeneous sensors and plat- cation and rate of operation to the developer, while forms. Scalar and multimedia sensor networks hiding the mechanisms of policy enforcement. should coexist in the same architecture, without compromising on performance; 6.4. Open research issues• Scalable. • While theoretical results on Slepian–Wolf and There is an inherent trade-oﬀ between degrees of Wyner–Ziv coding exist since 30 years, there isﬂexibility and network performance. Platform-inde- still a lack of practical solutions. The net beneﬁtspendence is usually achieved through layers of and the practicality of these techniques still needabstraction, which usually introduce redundancy to be demonstrated.and prevent the developer from accessing low-level • It is necessary to fully explore the trade-oﬀsdetails and functionalities. However, WMSNs are between the achieved ﬁdelity in the descriptioncharacterized by the contrasting objectives of opti- of the phenomenon observed, and the resultingmizing the use of the scarce network resources and energy consumption. As an example, the video
938 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 distortion perceived by the ﬁnal user depends on from temporary disruption of the application, it source coding (frame rate, quantization), and may cause rapid depletion of the node’s energy. on channel coding strength. For example, in a While applications running on traditional wire- surveillance application, the objective of maxi- less networks may only experience performance mizing the event detection probability is in con- degradation, the energy loss (due to collisions trast with the objective of minimizing the power and retransmissions) can result in network parti- consumption. tion. Thus, congestion control algorithms may• As discussed above, there is a need for high- need to be tuned for immediate response and layer abstractions that will allow fast develop- yet avoid oscillations of data rate along the ment of sensor applications. However, due to the aﬀected path. resource-constrained nature of sensor systems, • Packet re-ordering due to multi-path. Multiple it is necessary to control the eﬃciency of the paths may exist between a given source-sink pair, low-level operations performed on battery- and the order of packet delivery is strongly inﬂu- limited and resource-constrained sensor nodes. enced by the characteristics of the route chosen.• There is a need for simple yet expressive As an additional challenge, in real-time video/ high-level primitives for applications to leverage audio feeds or streaming media, information that collaborative, advanced in-network multimedia cannot be used in the proper sequence becomes processing techniques. redundant, thus stressing on the need for trans- port layer packet reordering.7. Transport layer We next explore the functionalities and support In applications involving high-rate data, the provided by the transport layer to address thesetransport layer assumes special importance by pro- and other challenges of WMSNs. The following dis-viding end-to-end reliability and congestion control cussion is classiﬁed into (1) TCP/UDP and TCPmechanisms. Particularly, in WMSNs, the following friendly schemes for WMSNs, and (2) application-additional considerations are in order to accommo- speciﬁc and non-standardized protocols. Fig. 7 sum-date both the unique characteristics of the WSN marizes the discussion in this section.paradigm and multimedia transport requirements. 7.1. TCP/UDP and TCP friendly schemes for• Eﬀects of congestion. In WMSNs, the eﬀect of WMSNs congestion may be even more pronounced as compared to traditional networks. When a bot- For real-time applications like streaming media, tleneck sensor is swamped with packets coming the User Datagram Protocol (UDP) is preferred from several high-rate multimedia streams, apart over TCP as timeliness is of greater concern than Transport Layer TCP/UDP and TCP Friendly Schemes Application Specific and Non-standard Protocols • TCP may be preferred over UDP unlike traditional wireless networks • Compatible with the TCP rate control mechanism Reliability Congestion Control Use of Multipath E.g.. STCP , MPEG-TFRCP  • Per-packet delivery • Spatio-temporal • Better load guarantee for selected reporting balancing and packet types robustness to channel • Adjusting of reporting state variability. • Redundancy by frequency based on caching at intermediate current congestion levels • Need to regulate nodes multiple sources Eg. ESRT  monitoring the same Eg. RMST , event PSFQ , (RT)2  Eg. CODA , MRTP  Fig. 7. Classiﬁcation of existing transport layer protocols.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 939reliability. However, in WMSNs, it is expected that can be, however, mitigated to a large extent bypackets are signiﬁcantly compressed at the source playout buﬀers at the sink, which is typicallyand redundancy is reduced as far as possible owing assumed to be rich in resources. As an example,to the high transmission overhead in the energy- the MPEG-TFRCP (TCP Friendly Rate Controlconstrained nodes. Under these conditions, we note Protocol for MPEG-2 Video Transfer)  is anthe following important characteristics that may equation-based rate control scheme designed fornecessitate an approach very diﬀerent from classical transporting MPEG video in a TCP-friendlywireless networks. manner. • Overhead of the reliability mechanism in TCP. As• Eﬀect of dropping packets in UDP. Simply drop- discussed earlier, blind dropping of packets in ping packets during congestion conditions, as UDP containing highly compressed video/audio undertaken in UDP, may introduce discernable data may adversely aﬀect the quality of transmis- disruptions in the order of a fraction of a second. sion. Yet, at the same time, the reliability mecha- This eﬀect is even more pronounced if the packet nism provided by TCP introduces an end-to-end dropped contains important original content not message passing overhead and energy eﬃciency captured by inter-frame interpolation, like the must also be considered. Distributed TCP Cach- Region of Interest (ROI) feature used in ing (DTC)  overcomes these problems by JPEG2000  or the I-frame used in the MPEG caching TCP segments inside the sensor network family. and by local retransmission of TCP segments.• Support for traﬃc heterogeneity. Multimedia traf- The nodes closest to the sink are the last-hop for- ﬁc comprising of video, audio, and still images warders on most of the high-rate data paths and exhibits a high level of heterogeneity and may thus run out of energy ﬁrst. DTC shifts the bur- be further classiﬁed into periodic or event driven. den of the energy consumption from nodes close The UDP header has no provision to allow any to the sink into the network, apart from reducing description of these traﬃc classes that may inﬂu- network wide retransmissions. ence congestion control policies. As a contrast to • Regulating streaming through multiple TCP con- this, the options ﬁeld in the TCP header can be nections. The availability of multiple paths modiﬁed to carry data speciﬁc information. As between source and sink can be exploited by an example, the Sensor Transmission Control opening multiple TCP connections for multime- Protocol (STCP)  accommodates a diﬀerenti- dia traﬃc . Here, the desired streaming rate ated approach by including relevant ﬁelds in the and the allowed throughput reduction in presence TCP header. Several other major changes to the of bursty traﬃc, like sending of video data, is traditional TCP model are proposed in the round communicated to the receiver by the sender. This trip time estimation, congestion notiﬁcation, the information is used by the receiver which then packet drop policy and by introducing a reliabil- measures the actual throughput and controls the ity driven intimation of lost packets. rate within the allowed bounds by using multiple TCP connections and dynamically changing its We thus believe that TCP with appropriate mod- TCP window size for each connection.iﬁcations is preferable over UDP for WMSNs, ifstandardized protocols are to be used. With respect TCP protocols tailor-made for wireless sensorto sensor networks, several problems and their networks is an active research area with recentlikely solutions like large TCP header size, data vs. implementations of the light-weight Sensor Internetaddress centric routing, energy eﬃciency, amongst Protocol (SIP)  and the open source uIP  thatothers, are identiﬁed and solutions are proposed in has a code size of few Kbyte. However, a major. We next indicate the recent work in this direc- problem with the TCP-based approach in wirelesstion that evaluates the case for using TCP in networks is its inability to distinguish between badWMSNs. channel conditions and network congestion. This has motivated a new family of specialized transport• Eﬀect of jitter induced by TCP. A key factor that layer where the design practices followed are limits multimedia transport based on TCP, and entirely opposite to that of TCP , or stress on TCP-like rate control schemes, is the jitter intro- a particular functionality of the transport layer, like duced by the congestion control mechanism. This reliability or congestion control.
940 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–9607.2. Application speciﬁc and non-standard protocols and in the order of 500 kbit/s , thus making it clear that congestion must be addressed in WMSNs. Depending on the application, both reliability While these data generation rates are high for a sin-and congestion control may be equally important gle node, multiple sensors in overlapped regionsfunctionalities or one may be preferred over the may inject similar traﬃc on sensing the same phe-other. As an example, in the CYCLOPS image cap- nomenon. The Event-to-Sink Reliable Transportturing and inference module  designed for (ESRT) protocol  leverages the fact that spatialextremely light-weight imaging, congestion control and temporal correlation exists among the individ-would be the primary functionality with multiple ual sensor readings . The ESRT protocol regu-sensor ﬂows arriving at the sink, each being moder- lates the frequency of event reporting in a remoteately loss-tolerant. We next list the important char- neighborhood to avoid congestions in the network.acteristics of such TCP incompatible protocols in However, this approach may not be viable for allcontext of WMSNs. sensor applications as nodes transmit data only when they detect an event, which may be a short7.2.1. Reliability duration burst as in the case of a video monitoring Multimedia streams may consist of images, video application. The feedback from the base-stationand audio data, each of which merits a diﬀerent may hence not reach in time to prevent a suddenmetric for reliability. As discussed in Section 7.1, congestion due to this burst.when an image or video is sent with diﬀerentiallycoded packets, the arrival of the packets with the 7.2.3. Use of multi-pathROI ﬁeld or the I-frame respectively should be guar- We advocate the use of multiple paths for dataanteed. The application can, however, withstand transfer in WMSNs owing to the following twomoderate loss for the other packets containing dif- reasons:ferential information. Thus, we believe that reliabil-ity needs to be enforced on a per-packet basis to • A large burst of data (say, resulting from anbest utilize the existing networking resources. If a I-frame) can be split into several smaller bursts,prior recorded video is being sent to the sink, all thus not overwhelming the limited buﬀers at thethe I-frames could be separated and the transport intermediate sensor nodes.protocol should ensure that each of these reach • The channel conditions may not permit high datathe sink. Reliable Multi-Segment Transport rate for the entire duration of the event being(RMST)  or the Pump Slowly Fetch Quickly monitored. By allowing multiple ﬂows, the eﬀec-(PSFQ) protocol  can be used for this purpose tive data rate at each path gets reduced and theas they buﬀer packets at intermediate nodes, allow- application can be supported.ing for faster retransmission in case of packetloss. However, there is an overhead of using the The design of a multiple source-sink transportlimited buﬀer space at a given sensor node for protocol is challenging, and is addressed by thecaching packets destined for other nodes, as well COngestion Detection and Avoidance (CODA)as performing timely storage and ﬂushing opera- protocol . It allows a sink to regulate multipletions on the buﬀer. In a heterogeneous network, sources associated with a single event in case of per-where real-time data is used by actors as discussed sistent network congestion. However, as the conges-in Section 4.3, the Real-time and Reliable Transport tion inference in CODA is based on queue length at(RT)2 protocol  can be used that deﬁnes diﬀerent intermediate nodes, any action taken by the sourcereliability constraints for sensor–actor and actor– occurs only after a considerable time delay. Otheractor communication. solutions include the Multi-ﬂow Real-time Trans- port Protocol (MRTP)  that does not speciﬁcally7.2.2. Congestion control address energy eﬃciency considerations in WMSNs, The high rate of injection of multimedia packets but is suited for real-time streaming of multimediainto the network causes resources to be used up content by splitting packets over diﬀerent ﬂows.quickly. While typical transmission rates for sensor MRTP does not have any mechanism for packetnodes may be about 40 kbit/s, indicative data rates retransmission and is mainly used for real-time dataof a constant bit rate voice traﬃc may be 64 kbit/s. transmission and hence, reliability can be an issueVideo traﬃc, on the other hand, may be bursty for scalar data traﬃc.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 9417.3. Open research issues like conﬁguration or initialization parameters, time-critical low rate data like presence or absence In summary, the transport layer mechanisms that of the sensed phenomenon, high bandwidth video/can simultaneously address the unique challenges audio data, etc. Each of the traﬃc classes describedposed by the WMSN paradigm and multimedia in Section 6.1 has its own QoS requirement whichcommunication requirements must be incorporated. must be accommodated in the network layer.While several approaches were discussed, some Research on the network layer becomes impor-open issues remain and are outlined below: tant from the standpoint of supporting multimedia applications constrained by lack of global knowl-• Trade-oﬀ between reliability and congestion con- edge, reduced energy, and computational ability of trol. In WMSN applications, the data gathered the individual nodes. from the ﬁeld may contain multimedia informa- We next discuss the existing research directions tion such as target images, acoustic signal, and for the network layer functionalities of addressing even video captures of a moving target, all of and routing, while stressing on their applicability which enjoy a permissible level of loss tolerance. to delay-sensitive and high bandwidth needs. Presence or absence of an intruder, however, may require a single data ﬁeld but needs to be commu- 8.1. Addressing and localization nicated without any loss of ﬁdelity. Thus, when a single network contains multimedia as well as In the case of large WMSNs like IrisNet , it is scalar data, the transport protocol must decide required that the individual nodes be monitored via whether to focus on one or more functionalities the Internet. Such an integration between a ran- so that the application needs are met without domly deployed sensor network and the established an unwarranted energy expense. The design of wired network becomes a diﬃcult research chal- such a layer may as well be modular, with the lenge. The key problem of global addressing could functional blocks of reliability and/or congestion be solved by the use of IPv6 in which the sensor control being invoked as per network demands. can concatenate its cluster ID with its own MAC• Real-time communication support. Despite the address to create the full IPv6 address. However, existence of reliable transport solutions for the 16-byte address ﬁeld of IPv6 introduces exces- WSN as discussed above, none of these protocols sive overhead in each sensor data packet. There provide real-time communication support for the are several other schemes that assign unique net- applications with strict delay bounds. Therefore, work-wide IDs (see  and references therein) or new transport solutions which can also meet cer- leverage location information to create an address- tain application deadlines must be researched. free environment but they, however, run the risk• Relation between multimedia coding rate and of incompatibility with the established standards reliability. The success in energy-eﬃcient and of the Internet. Location information is a key char- reliable delivery of multimedia information acteristic of any sensor network system. The ability extracted from the phenomenon directly depends to associate localization information to the raw data on selecting appropriate coding rate, number of sampled from the environment increases the capa- sensor nodes, and data rate for a given event bility of the system and the meaningfulness of the . However, to this end, the event reliability information extracted. Localization techniques for should be accurately measured in order to eﬃ- WMSNs are unlikely to diﬀer substantially from ciently adapt the multimedia coding and trans- those developed for traditional sensor networks, mission rates. For this purpose, new reliability which are reviewed in . Moreover, WMSNs metrics coupled with the application layer coding will most likely leverage the accurate ranging capa- techniques should be investigated. bilities that come with high bandwidth transmis- sions (such as UWB techniques, as described in Section 10).8. Network layer 8.2. Routing The network layer addresses the challenging taskof providing variable QoS guarantees depending on Data collected by the sensor nodes needs to bewhether the stream carries time-independent data sent to the sink, where useful information can be
942 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 Routing Layer Network Condition Based Traffic Classes Based Real Time Streaming Based    Metrics - Metrics - Metrics - • Position wrt sink • QoS profiles/Traffic classes • Spatio-temporal character • Radio characteristics • Dropping rate • Probabilistic delay guarantees • Error rate • Latency tolerance • Residual energy • Desired bandwidth • Backlogged packets Fig. 8. Classiﬁcation of existing routing protocols.extracted from it. Comprehensive surveys of the This approach involves an overhead in whichmajor routing schemes existing in the literature are nodes must apprise their neighbors of any changespresented in [19,23]. The concerns of routing in gen- in the cost function parameters. This work alsoeral diﬀer signiﬁcantly from the specialized service deals with relative priority levels for event basedrequirements of multimedia streaming applications. (high bandwidth) and periodic (low bandwidth)As an example, multiple routes may be necessary to data.satisfy the desired data rate at the destination node. A similar scenario is considered in  whereAlso, diﬀerent paths exhibiting varying channel con- imaging data for sensor networks results in QoSditions may be preferred depending on the type of considerations for routing, apart from the tradi-traﬃc and its resilience to packet loss. We next dis- tional goal of energy conservation. Here, the costcuss the various approaches to routing in WMSNs function evaluates the residual energy of a node,while maintaining that protocols may incorporate transmission energy, error rate and other communi-features from more than one of the following clas- cation parameters. The protocol ﬁnds a least-cost,ses. Majorly, they can be classiﬁed into routing energy eﬃcient path while considering maximumbased on (i) network conditions that leverage chan- allowed delays.nel and link statistics, (ii) traﬃc classes that decidepaths based on packet priorities, and (iii) specialized 8.2.2. QoS routing based on traﬃc classesprotocols for real-time streaming that use spatio- Sensor data may originate from various types oftemporal forwarding. Fig. 8 provides a classiﬁcation events that have diﬀerent levels of importance, asof existing routing protocols and summarizes the described in Section 6.1. Consequently, the contentdiscussion in this section. and nature of the sensed data also varies. As an example that highlights the need for network level8.2.1. QoS routing based on network conditions QoS, consider the task of bandwidth assignment Network conditions include interference seen at for multimedia mobile medical calls, which includeintermediate hops, the number of backlogged ﬂows patients’ sensing data, voice, pictures and video dataalong a path, residual energy of the nodes, amongst . Unlike the typical source-to-sink multi-hopothers. A routing decision based on these metrics communication used by classical sensor networks,can avoid paths that may not support high band- the proposed architecture uses a 3G cellular systemwidth applications or introduce retransmission in which individual nodes forward the sensed dataowing to bad channel conditions. to a cellular phone or a specialized information col- The use of image sensors is explored in , in lecting entity. Diﬀerent priorities are assigned towhich visual information is used to gather topology video data originating from sensors on ambulances,information that is then leveraged to develop eﬃ- audio traﬃc from elderly people, and imagescient geographic routing schemes. A weighted cost returned by sensors placed on the body. In orderfunction is constructed that takes into account posi- to achieve this, parameters like hand-oﬀ droppingtion with respect to the base station, backlogged rate (HDR), latency tolerance and desired amountpackets in the queue, and remaining energy of of wireless eﬀective bandwidth are taken intothe nodes to decide the next hop along a route. consideration.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 9438.2.3. Routing protocols with support for streaming decision making process and network tuning The SPEED protocol  provides three types from the user end into the network.of real-time communication services, namely, real- • As the connectivity between diﬀerent domainstime unicast, real-time area-multicast and real-time improves, end-to-end QOS guarantees arearea-anycast. It uses geographical location for complicated by the inherent diﬀerences in therouting and a key diﬀerence with other schemes of nature of the wired and wireless media. Whenthis genre is its spatio-temporal character, i.e., it sensed data from the ﬁeld is sent via the Internet,takes into account timely delivery of the packets. a single routing metric is unsuitable for theIt is speciﬁcally tailored to be a stateless, localized entire path between source and end user. Decou-algorithm with minimal control overhead. End- pling of reliability and routing parameters atto-end soft real-time communication is achieved such network boundaries and a seamless inte-by maintaining a desired delivery speed across gration of schemes better suited to wired orthe sensor network through a combination of wireless domains, respectively, will need to befeedback control and non-deterministic geographic explored.forwarding. As it works satisfactorily underscarce resource conditions and can provide service 9. MAC layerdiﬀerentiation, SPEED takes the ﬁrst step inaddressing the concerns of real-time routing in Owing to the energy constraints of the small, bat-WMSNs. tery-powered sensor nodes, it is desirable that the A signiﬁcant extension over SPEED, the medium access control (MAC) protocol enableMMSPEED protocol  can eﬃciently diﬀerenti- reliable, error-free data transfer with minimumate between ﬂows with diﬀerent delay and reliability retransmissions while supporting application-spe-requirements. MMSPEED is based on a cross-layer ciﬁc QoS requirements. Multimedia traﬃc, namelyapproach between the network and the MAC layers audio, video, and still images can be classiﬁed asin which a judicious choice is made over reliability separate service classes and subjected to diﬀerentand timeliness of packet arrival. It is argued that policies of buﬀering, scheduling and transmission.the diﬀerentiation in reliability is an eﬀective way The need for packet-speciﬁc diﬀerentiation is justi-of channeling resources from ﬂows with relaxed ﬁed in the context of the following example. Therequirements to ﬂows with tighter requirements. new standard for the compression of still images,Importantly, a new metric called On-Time Reach- JPEG2000 , incorporates a feature called regionability is introduced which is a measure of the prob- of interest (ROI) that may be applicable to visualability that a packet reaches its destination within data sensing. It allows the allocation of greaterrequired delay bounds. While current research importance to certain parts of the image whichdirections make an eﬀort to provide real-time can then be coded and transmitted over a betterstreaming, they are still best eﬀort services. Giving quality link or on a priority basis. Especially rele-ﬁrm delay guarantees in a dynamically changing vant to systems for military surveillance or faultnetwork is a diﬃcult problem and yet is important monitoring, such application layer features couldfor seamless viewing of the multimedia frames. be leveraged by the MAC by diﬀerentially treatingMMSPEED takes the step towards this end by the ROI packets.adopting a probabilistic approach but clearly, fur- Research eﬀorts to provide MAC layer QoS canther work is needed in this area. be classiﬁed mainly into (i) channel access policies, (ii) scheduling and buﬀer management, and (iii)8.3. Open research issues error control. We next provide a brief description of each and highlight their support to multimedia• The identiﬁcation of the optimal routing metrics traﬃc. The scope of this paper is limited to the chal- is a continual area of research. Most routing pro- lenges posed by multimedia traﬃc in sensor net- tocols that consider more than one metric, like works and the eﬀorts at the MAC layer to address energy, delay etc., form a cost function that is them. A detailed survey of MAC protocols for clas- then minimized. The choice of the weights for sical sensor networks using scalar data can be found these metrics need to be judiciously undertaken, in . Fig. 9 provides a classiﬁcation of relevant and is often subject to dynamic network condi- MAC layer functionalities and summarizes the dis- tions. Thus, further work is needed to shift this cussion in this section.
944 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 Medium Access Control Contention Free Contention Based • Coordinate sleep/awake cycles • Bursty nature of scheduling may lead to jitters Single Channel Multi-channel E.g.. S-MAC , T-MAC  • TDMA -like • Better bandwidth utilization • Better control for multimedia • Hardware assumptions design parameters • Channel switching delay may be a • Simple hardware, operation consideration in end to end latency • MIMO technology E.g.. STEM , RATE-EST , CMAC  E.g.. STE , EDD  Fig. 9. Classiﬁcation of protocols at the data link layer.9.1. Channel access policies gation in light of the ongoing high data rate video/audio messaging. The main causes of energy loss in sensor net- • Video traﬃc exhibits an inherent bursty natureworks are attributed to packet collisions and subse- and can lead to sudden buﬀer overﬂow at thequent retransmissions, overhearing packets destined receiver. This problem is further aggravated byfor other nodes, and idle listening, a state in which the transmission policy adopted in T-MAC .the transceiver circuits remain active even in the By choosing to send a burst of data duringabsence of data transfer. Thus, regulating access the listen cycle, T-MAC shows performanceto the channel assumes primary importance and sev- improvement over S-MAC, but at the cost oferal solutions have been proposed in the literature. monopolizing a bottleneck node. Such an opera- tion could well lead to strong jitters and result in9.1.1. Contention-based protocols discontinuous real-time playback. Most contention-based protocols like S-MAC, and protocols inspired by it , have a sin-gle-radio architecture. They alternate between sleep 9.1.2. Contention-free single channel protocolscycles (low-power modes with transceiver switched Time Division Multiple Access (TDMA) is aoﬀ) and listen cycles (for channel contention and representative protocol of this class in which thedata transmission). However, we believe that their clusterhead (CH) or sink helps in slot assignment,applicability to multimedia transmission is limited querying particular sensors and maintaining timeowing to the following reasons: schedules. We believe that such protocols can be easily adapted for multimedia transmission and• The primary concern in the protocols of this class highlight the likely design considerations. is saving energy, and this is accomplished at the cost of latency and by allowing throughput deg- • TDMA schemes designed exclusively for sensor radation. A sophisticated duty cycle calculation networks  (and references therein) have a based on permissible end-to-end delay needs to small reservation period (RP) that is generally be implemented and coordinating overlapping contention based, followed by a contention-free listen period with neighbors based on this calcu- period that spans the rest of the frame. This RP lation is a diﬃcult research challenge. could occur in each frame or at pre-decided inter-• Coordinating the sleep–awake cycles between vals in order to assign slots to active nodes taking neighbors is generally accomplished though sche- into consideration the QoS requirement of their dule exchanges. In case of dynamic duty cycles data streams. The length of the TDMA frames based on perceived values of instantaneous or and the frequency of the RP interval are some time averaged end-to-end latency, the overhead of the design parameters that can be exploited of passing frequent schedules also needs investi- while designing a multimedia system.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 945• For real-time streaming video, packets are time to both the sender and receiver at an acceptable constrained and scheduling policies like Shortest energy cost. Time to Extinction (STE)  or Earliest Due Date (EDD)  can be adopted. Both of these 126.96.36.199. Open research issues are similar in principle as packets are sent in the increasing order of their respective delay tol- • While TDMA schedules within a cluster can be erance but diﬀer in respect that EDD may still easily devised, the problem is more involved forward a packet that has crossed its allowed when individual CHs are not in direct range of delay bound. Based on the allowed packet loss the sink, thus necessitating inter-cluster multi- of the multimedia stream, the dependencies hop communication. An acceptable, non-over- between packet dropping rate, arrival rate, and lapping slot assignment for all neighboring delay tolerance  can be used to decide the clusters needs to be derived in a distributed man- TDMA frame structure and thus ensure smooth ner requiring coordination between them at the replay of data. This allows greater design choices set-up phase. This problem has been shown to as against , where the frame lengths and slot be NP-complete  by reduction to an instance duration are considered constant. of graph coloring and the development of eﬃ-• As sensor nodes are often limited by their maxi- cient heuristics is an open issue. mum data transmission rate, depending upon • The eﬀect of clock drift is pronounced if the slot their multimedia traﬃc class, the duration of duration is small and rigid time synchronization transmission could be made variable. Thus vari- is required for best performance  (and refer- able TDMA (V-TDMA) schemes should be pre- ences therein). Network scalability is another ferred when heterogeneous traﬃc is present in the important area of research and the TDMA network. Tools for calculating the minimum schedules must be able to accommodate high worst-case delay in such schemes and algorithms node densities that are characteristic of sensor for link scheduling are provided in . As real- networks. As channel capacity in TDMA is ﬁxed, time streaming media is delay bounded, the only slot durations or number of slots in a frame link-layer latency introduced in a given ﬂow may be changed keeping in mind the number of due in order to satisfy data rate requirements of users and their respective traﬃc types. another ﬂow needs to be analyzed well when V- • Bounds on unequal slot/frame lengths for diﬀer- TDMA schemes are used. entiated services should be decided by the allowed per-hop delay (and consequently end- to-end delay). Schedules, once created, should188.8.131.52. MIMO technology. The high data rate also be able to account for a dynamically chang-required by multimedia applications can be ing topology due to nodes dying oﬀ or new onesaddressed by spatial multiplexing in MIMO sys- being added.tems, that use a single channel but employ interfer-ence cancellation techniques. Recently, virtualMIMO schemes have been proposed for sensor net- 9.1.3. Contention-free multi-channel protocolsworks , where nodes in close proximity form a Clearly, existing data rates of about 40 kbit/s andcluster. Each sensor functions as a single antenna 250 kbit/s supported by the MICA2 and MICAzelement, sharing information and thus simulating motes are not geared to support multimedia traﬃc.the operation of a multiple antenna array. A distrib- Along with improving hardware and thus increasinguted compression scheme for correlated sensor data, cost, an alternate approach is to eﬃciently utilizethat specially addresses multimedia requirements, is the available bandwidth. By using multiple channelsintegrated with the MIMO framework in . How- in a spatially overlapped manner, existing band-ever, a key consideration in MIMO based systems is width can be eﬃciently utilized for supporting mul-the number of sensor transmissions and the required timedia applications. We observe that Berkeley’ssignal energy per transmission. As the complexity is third generation MICA2 Mote has an 868/shifted from hardware to sensor coordination, fur- 916 GHz multi-channel transceiver . In Rock-ther research is needed at the MAC layer to ensure well’s WINS nodes, the radio operates on one ofthat the required MIMO parameters like channel 40 channels in the ISM frequency band, selectablestate, desired diversity/processing gain are known by the controller . We next outline the design
946 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960parameters that could inﬂuence MAC design in • Recently, the cognitive radio paradigm [21,55]multi-channel WMSNs. has drawn signiﬁcant interest in which the radio is aware of its surroundings, learns from it and• Recent research has focused on a two-transceiver adapts in order to give the best possible function- paradigm in which the main radio (MR) is supple- ality to the user. By dynamically sharing spectrum mented by the presence of a wake-up radio (LR) with other services, unused frequency bands can having similar characteristics [88,114,115] or a be utilized ensuring optimum bandwidth usage. simple low-energy design  that emits a series The release of new spectrum in the 5-GHz U- of short pulses or a busy tone. The LR is used NII band has spurred on interest in realizing a for exchanging control messages and is assigned practical system for multimedia applications. a dedicated channel. In high bandwidth applica- However, porting it to a low-power sensor node tions, like streaming video, the use of a separate and developing controlling mechanisms for chan- channel for channel arbitration alone does not nel hand-oﬀ is an area that is yet to be explored. allow best utilization of the network resources. • Multi-channel protocols are not completely colli-• We propose that WMSNs use in-band signalling, sion free as is seen in the case of control packet where the same channel is used for both data and collision [33,88,114]. All available channels can- channel arbitration . While such protocols not be assumed to be perfectly non-overlapping, undoubtedly improve bandwidth eﬃciency, they as is seen in the case of 802.11b based WLANs introduce the problem of distinct channel assign- . This may necessitate dynamic channel ment and need to account for the delay to switch assignment, taking into account the eﬀect of to a diﬀerent channel , as its cumulative nat- adjacent channel interference, in order to main- ure at each hop aﬀects real-time media. tain the network QoS.• Existing compression techniques like JPEG and MPEG cause size variations of the captured video frame depending upon the movement and 9.2. Scheduling compression rate within a given frame, and the subsequent following frames. Thus, the arrival MAC layer scheduling in the context of WMSNs packet rate (assuming packet size constant) may diﬀers from the traditional networking model in the change over time and, we believe, enforcing sense that apart from choosing the queueing disci- wake-ups based on the packet backlog is a better pline that accounts for latency bounds, rate/power design approach as compared to those with static control and consideration of high channel error timer based schemes. A dual approach is fol- conditions needs to be incorporated. We believe lowed in , but this, however, raises questions that an optimal solution is a function of all of these of receiver side synchronization as knowledge of factors, appropriately weighted and seamlessly inte- the sender’s buﬀer state and its consequent sched- grated with a suitable channel access policy uling instant is unknown at the receiver end. described in Section 9.1. In order to generate optimal schedules that mini- mize both power consumption and the probability of184.108.40.206. Open research issues missing deadlines for real-time messages, PARM  integrates the EDD metric described in Section• Multi-channel protocols utilize bandwidth better, 9.1.2 into an energy consumption function. While and thus may perform favorably in cases of signiﬁcant performance improvements are demon- applications demanding high data rate. The strated, this work needs to be extended for large-scale results obtained in , leave an open question networks that are typically envisaged for WMSNs. on whether switching delay can be successfully Queueing at the MAC layer has been extensively hidden with only one interface per node. If this researched and several schemes with varying levels is possible, it may greatly simplify sensor design of complexity exist. Of interest to multimedia appli- while performing as well as a multi-channel, cations is the development of schemes that allow a multi-interface solution. Also, the sleep–awake delay bound and thus assure smooth streaming of schedule of the radios should be made dynamic multimedia content. E2WFQ , a variant of the in order to accommodate the varying frame rate established weighted fair queuing (WFQ) discipline, for video sensors. allows adjustments to be made to the energy-
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 947latency-ﬁdelity trade-oﬀ space. Depending upon the shadowing at the physical layer, and by collisionscurrent residual energy in the network, it is possible or co-channel interference at the MAC layer. Twoto adapt the scheme for greater energy savings, main classes of mechanisms are traditionallyalbeit at the cost of a small, bounded increase in employed to combat the unreliability of the wirelessworst-case packet latency. channel at the physical and data link layer, namely Given the bursty nature of voice and video data, forward error correction (FEC) and automatic repeatqueueing disciplines are needed that can accommo- request (ARQ), along with hybrid schemes. ARQdate sudden peaks, as well as operate under local- mechanisms use bandwidth eﬃciently at the costized channel errors. Extending WFQ, the Wireless of additional latency. Hence, while carefullyPacket Scheduling (WPS) presented in , designed selective repeat schemes may be of someaddresses the concerns of delay and rate-sensitive interest, naive use of ARQ techniques is clearlypacket ﬂows thus making it suitable for multimedia infeasible for applications requiring real-time deliv-traﬃc. WPS, however, assumes that the channel ery of multimedia content.error is fully predictable at any time and its practical An important characteristic of multimedia con-implementation shows marked deviations from the tent is unequal importance, i.e., not all packets haveidealized case in terms of worst-case complexity. the same importance for correct perceptual recon-This work is suitable for single-hop sensor-sink struction of the multimedia content. Moreover,communication and multi-hop forwarding issues multimedia data are usually error-tolerant, so evenare not explored. if some errors are introduced, the original informa- Network calculus [27,36] is a theory for deter- tion may still be reconstructed with tolerable distor-ministic queueing systems that allows assignment tion. Therefore, an idea that has been usedof service guarantees by traﬃc regulation and deter- eﬀectively consists of applying diﬀerent degrees ofministic scheduling. Through tools provided by net- FEC to diﬀerent parts of the video stream, depend-work calculus, bounds on various performance ing on their relative importance (unequal protection).measures, such as delay and queue length, at each For example, this idea can be applied to layeredelement of the network can be derived and thus coded streams to provide graceful degradation inQoS of a ﬂow can be speciﬁed. Arrival, Departure the observed image quality in presence of errorand Service curves reﬂect the constraints that ﬂows losses, thus avoiding so-called ‘‘cliﬀ’’ eﬀects .are subjected to within a network. The calculus In general, delivering error-resilient multimediarelies on Min-plus algebra, in which addition and content and minimizing energy consumption aremultiplication are replaced by minimum and addi- contradicting objectives. For this reason, and duetion, respectively, to operate on these curves. Cur- to the time-varying characteristics of the wirelessrent network calculus results have been mostly channel, several joint source and channel codingderived for wired networks, and assume static topol- schemes have been developed, e.g. , which tryogies and ﬁxed link capacity, which are clearly to reduce the energy consumption of the whole pro-unreasonable assumptions in sensor networks. We cess. Some recent papers [77,135] even try to jointlybelieve that extending network calculus results to reduce the energy consumption of the whole processWMSNs is a challenging but promising research of multimedia content delivery, i.e., jointly optimizethrust, likely to produce important advancements source coding, channel coding, and transmissionin our ability to provide provable QoS guarantees power control. In these schemes, the image codingin multi-hop networks. and transmission strategies are adaptively adjusted to match current channel conditions by exploiting9.3. Link-layer error control the peculiar characteristics of multimedia data, such as unequal importance of diﬀerent frames or layers. Streaming of real-time multimedia data over a However, most of these eﬀorts have originated fromsensor network is particularly challenging due to the multimedia or coding communities, and thus dothe QoS requirements of a video/audio stream and not jointly consider other important networkingthe unreliability of the wireless medium. For exam- aspects of content delivery over a multi-hop wirelessple, for good quality video perception a frame loss networks of memory-, processing- and battery-con-rate lower than 10À2 is required. This constitutes a strained devices.hard task since the wireless channel is highly unreli- In , a cross-layer analysis of error controlable, mostly caused by multi-path fading and schemes for WSNs is presented. The eﬀects of
948 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960multi-hop routing and of the broadcast nature of • Since multimedia data is usually error-tolerant,wireless communications are investigated to model new packet dropping schemes for multimediathe energy consumption, latency and packet error delivery have to be delivered, that selectivelyrate performance of error control schemes. As a drop packets that will not impact the perceivedresult, error control schemes are studied through a quality at the end user.cross-layer analysis that considers the eﬀects of • Energy-constrained sensor networks naturallyrouting, medium access, and physical layer. This call for Selective Repeat ARQ techniques. How-analysis enables a comprehensive comparison of ever, this can introduce excessive latency. ThereFEC and ARQ schemes in WSNs. FEC schemes is a need to study trade-oﬀs between the degreeare shown to improve the error resiliency compared of reliability required (i.e., acceptable packetto ARQ. In a multi-hop network, this improvement error rate) and the sustainable delay at the appli-can be exploited by reducing the transmit power cation layer.(transmit power control) or by constructing longer • Coordinating link and transport layer error recov-hops (hop length extension) through channel-aware ery schemes is a challenge that remains to berouting protocols. The analysis reveals that, for cer- addressed. In order to ensure that buﬀer over-ﬂowtain FEC codes, hop length extension decreases conditions do not occur, mechanisms that detectboth the energy consumption and the end-to-end increased MAC level contention and regulate datalatency subject to a target packet error rate com- generation rate could be implemented.pared to ARQ. Thus, FEC codes are an important • In-network storage schemes , in which data iscandidate for delay-sensitive traﬃc in WSNs. On stored within the sensor nodes itself and accessedthe other hand, transmit power control results in on demand, may further reduce available buﬀersigniﬁcant savings in energy consumption at the cost capacity of the sensor nodes. Thus allocation ofof increased latency. optimum buﬀer sizes truly merits a cross-layer A diﬀerent approach to link-layer reliability is approach, spanning application layer architec-proposed through the use of erasure codes . In ture to existing channel conditions seen at thesensor networks where message passing is mini- PHY layer.mized, such a scheme signiﬁcantly reduces the needfor retransmissions. It allows recovering of m origi- 9.4. Multimedia packet sizenal messages by receiving any m out of n codewords. In lossy conditions, the number of such code In wireless networks, the successful reception of awords generated can be optimized against the energy packet depends upon environmental factors thatexpense in their transmission to ensure greater reli- decide the bit error rate (BER) of the link.ability. In multimedia applications, where the loss Packet length clearly has a bearing on reliableof a few frames may be tolerated, shorter code link level communication and may be adjustedwords may be used. When detailed, higher resolu- according to application delay sensitivity require-tion images are needed, regenerating the lost infor- ments. The Dynamic Packet Size Mechanismmation through these codes may be preferred over (DPSM) scheme  for wireless networks followsinterpolation of the missing data at the receiver an additive increase, multiplicative decreaseend. It should be noted that this approach works (AIMD) mechanism to decide the packet length,best for static reliability requirements. Dynamically analogous to the congestion control performed bychanging code lengths also increases the packet size TCP at the transport layer. As an example, if aand its overall eﬀect on other factors like congestion packet fails the checksum, the sender is intimatedcannot be trivially estimated. and the subsequent packets are sent with a multipli- cative decrease in length. However, the problems9.3.1. Open research issues associated with burst-error channel and the adapta- tion of video quality have not been addressed in this• There is a need to develop models and algorithms work. Grouping smaller packets together in order to to integrate source and channel coding schemes reduce contention has been explored in Packet in existing cross-layer optimization frameworks. Frame Grouping (PFG)  and PAcket Concate- The existing schemes mostly consider point-to- nation (PAC) . Originally devised for 802.11- point wireless links, and neglect interference from like protocols, here the header overhead is shared neighboring devices and multi-hop routes. by the frames. In PFG, the individual frames may
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 949be addressed to diﬀerent senders and requires per- to speciﬁc services, UWB devices are allowed toframe ACKs while PAC requires buﬀering as all operate overlaid and thus interfere with existingframes need to have a common destination. Depend- services, at a low enough power level that existinging upon the information content of the frame and services would not experience performance degrada-the channel conditions, variable length forward tion. The First Report and Order by the FCCerror-correcting codes (FEC) can be used to reduce includes standards designed to ensure that existingthe eﬀects of transmission errors at the decoder. The and planned radio services, particularly safetytrade-oﬀ between the increase of packet length due services, are adequately protected.to the additional parity bits and energy constraints There exist two main variants of UWB. The ﬁrst,is evaluated in , where FEC is shown to per- known as Time-Hopping Impulse Radio UWBform better than retransmissions. (TH-IR-UWB) , and mainly developed by Win and Scholtz , is based on sending very short duration pulses (in the order of hundreds of10. Physical layer picoseconds) to convey information. Time is divided into frames, each of which is composed of several In this section, we discuss the applicability of the chips of very short duration. Each sender transmitsUWB transmission technique, which we advocate one pulse in a chip per frame only, and multi-userover other technologies such as Zigbee, as the most access is provided by pseudo-random time hoppingsuitable choice for multimedia sensors. sequences (THS) that determine in which chip each user should transmit. A diﬀerent approach, known10.1. Ultra wide band communications as Multi-Carrier UWB (MC-UWB), uses multiple simultaneous carriers, and is usually based on The ultra wide band (UWB)1 technology has the Orthogonal Frequency Division Multiplexingpotential to enable low-power consumption, high (OFDM) .data rate communications within tens of meters, MC-UWB is particularly well suited for avoidingcharacteristics that make it an ideal choice for interference because its carrier frequencies can beWMSNs. precisely chosen to avoid narrowband interference UWB signals have been used for several decades to or from narrowband systems. However, imple-in the radar community. Recently, the US Federal menting a MC-UWB front-end power ampliﬁerCommunications Commission (FCC) Notice of can be challenging due to the continuous variationsInquiry in 1998 and the First Report and Order in in power over a very wide bandwidth. Moreover,2002  inspired a renewed ﬂourish of research when OFDM is used, high-speed FFT processingand development eﬀorts in both academy and indus- is necessary, which requires signiﬁcant processingtry due to the characteristics of UWB that make it a power and leads to complex transceivers.viable candidate for wireless communications in TH-IR-UWB signals require fast switching timesdense multi-path environments. for the transmitter and receiver and highly precise Although UWB signals, as per the speciﬁcations synchronization. Transient properties becomeof the FCC, use the spectrum from 3.1 GHz to important in the design of the radio and antenna.10.6 GHz, with appropriate interference limitation, The high instantaneous power during the brief inter-UWB devices can operate using spectrum occupied val of the pulse helps to overcome interference toby existing radio services without causing interfer- UWB systems, but increases the possibility of inter-ence, thereby permitting scarce spectrum resources ference from UWB to narrowband systems. The RFto be used more eﬃciently. Instead of dividing the front-end of a TH-IR-UWB system may resemble aspectrum into distinct bands that are then allocated digital circuit, thus circumventing many of the prob- lems associated with mixed signal integrated cir- cuits. Simple TH-IR-UWB systems can be very 1 The FCC deﬁnes UWB as a signal with either a fractional inexpensive to construct.bandwidth of 20% of the center frequency or 500 MHz (when the Although no sound analytical or experimentalcenter frequency is above 6 GHz). The FCC calculates the comparison between the two technologies is avail-fractional bandwidth as 2(fH À fL)/(fH + fL) where fH representsthe upper frequency of the À10 dB emission limit and fL able to our knowledge, we believe that TH-IR-represents the lower frequency limit of the À10 dB emission limit UWB is particularly appealing for WMSNs for. the following reasons:
950 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960• It enables high data rate, very low-power wireless where c is the speed of light, SNR represents the communications, on simple-design, low-cost signal-to-noise ratio, and b is the eﬀective signal radios (carrierless, baseband communications) bandwidth. As can be seen, the accuracy of the . time-based localization technique can be improved• Its ﬁne delay resolution properties are appropri- by either increasing the eﬀective bandwidth or the ate for wireless communications in dense multi- SNR. For this reason, the large bandwidth of path environment, by exploiting more resolvable UWB systems allows extremely accurate location paths . estimations, e.g., within one inch at SNR = 0 dB• Provides large processing gain in presence of and with a pulse of 1.5 GHz bandwidth. Excellent interference. comprehensive surveys of the UWB transmission• Provides ﬂexibility, as data rate can be traded technique, and of localization techniques for UWB for power spectral density and multi-path systems, are provided in [132,49], respectively. performance.• Finding suitable codes for THS is trivial (as 10.1.2. Standards based on UWB opposed to CDMA codes), and no assignment The IEEE 802.15.3a task group has been discuss- protocol is necessary. ing for three years an alternate physical layer for its• It naturally allows for integrated MAC/PHY high data rate Wireless Personal Area Networks solutions; . Moreover, interference mitigation (WPAN) standard. However, in early 2005 the techniques  allow realizing MAC protocols group has been disbanded after not being able to that do not require mutual temporal exclusion reach a consensus on a single UWB-based standard between diﬀerent transmitters. In other words, between two competing proposal from two leading simultaneous communications of neighboring industry groups, the UWB Forum and the WiMedia devices are feasible without complex receivers Alliance. The UWB Forum proposal was based on a as required by CDMA. Direct Sequence DS-UWB technology, while the• The large instantaneous bandwidth enables ﬁne Wimedia alliance was proposing a Multi-band time resolution for accurate position estima- Orthogonal Frequency Division Multiplexing tion  and for network time distribution (MB-OFDM). The IEEE 802.15.4a task group is (synchronization). developing an alternate physical layer for low data• UWB signals have extremely low-power spectral rate, very low-power consumption sensors, based density, with low probability of intercept/detec- on impulse radio UWB. tion (LPI/D), which is particularly appealing for military covert operations. 10.1.3. Open research issues10.1.1. Ranging capabilities of UWB • While the UWB transmission technology is Particularly appealing for WMSNs are UWB advancing rapidly, many challenges need to behigh data rate with low-power consumption, and solved to enable multi-hop networks of UWBits positioning capabilities. Positioning capabilities devices. In particular, although some recentare needed in sensor networks to associate physical eﬀorts have been undertaken in this directionmeaning to the information gathered by sensors. [39,87], how to eﬃciently share the medium inMoreover, knowledge of the position of each net- UWB multi-hop networks is still an open issue.work device allows for scalable routing solutions • As a step ahead, research is needed aimed at. While angle-of-arrival techniques and signal designing a cross-layer communication architec-strength based techniques do not give particular ture based on UWB with the objective of reliablyadvantages with respect to other transmission tech- and ﬂexibly delivering QoS to heterogeneous appli-niques, time-based approaches in UWB allow rang- cations in WMSNs, by carefully leveraging anding accuracy in the order of centimeters . This controlling interactions among layers accordingcan be intuitively explained by the expression in to the applications requirements.(1), which gives a lower bound on the best achiev- • It is necessary to determine how to provide prov- ^able accuracy of a distance estimate d : able latency and throughput bounds to multime-qﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ dia ﬂows in an UWB environment. ^ c • It is needed to develop analytical models to quan- VarðdÞ P pﬃﬃﬃ pﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃﬃ ; ð1Þ 2 2p SNRb titatively compare diﬀerent variants of UWB to
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 951 determine trade-oﬀs in their applicability to high neous nodes necessitate some form of topology con- data rate and low-power consumption devices trol in order to derive optimum ratios of FFD and such as multimedia sensors. RFD devices. Such a ratio will be dependent on• A promising research direction may also be to the region being monitored, the desired coverage integrate UWB with advanced cognitive radio accuracy, amongst others. There is no built-in sup-  techniques to increase the spectrum utiliza- port in Zigbee that splits up large data into smaller tion. For example, UWB pulses could be adap- packets and additional code has to be inserted at the tively shaped to occupy portions of the spectrum application layer. Video and image capturing sen- that are subject to lower interference. sors will hence need a specialized PHY/MAC aware application layer.10.2. Other physical layer technologies 11. Cross-layer design Devices based on the speciﬁcations issued by the As previously discussed, in multi-hop wirelessZigbee alliance can ﬁnd applicability in low data networks there is a strict interdependence amongrate applications that require simple forms of QoS functions handled at all layers of the communica-guarantees. Zigbee  is a speciﬁcation for a suite tion stack. The physical, MAC, and routing layersof high-level communication protocols using small, together impact the contention for networklow-power digital radios based on the IEEE resources. The physical layer has a direct impact802.15.4 standard for wireless personal area net- on multiple access of nodes in wireless channels byworks (WPANs). IEEE 802.15.4 can operate at aﬀecting the interference at the receivers. Thebandwidths of 250 kbit/s at 2.4 GHz, 40 kbit/s at MAC layer determines the bandwidth allocated to915 MHz (America) and 20 kbit/s at 868 MHz each transmitter, which naturally aﬀects the perfor-(Europe). While the data rate is much lower than mance of the physical layer in terms of successfullyBluetooth, energy consumption is much lesser here detecting the desired signals. On the other hand, as aand recent low-cost commercial versions have result of transmission schedules, high packet delaysdemonstrated the viability of this technology for and/or low bandwidth can occur, forcing the rout-low duty cycle (<0.01) sensor applications. The ing layer to change its route decisions. DiﬀerentCSMA-CA MAC allows a large number of devices routing decisions alter the set of links to be sched-to be connected simultaneously while provision is uled, and thereby inﬂuence the performance of thealso made for guaranteed time slot communication. MAC layer. Furthermore, congestion control andThe IEEE 802.15.4 standard allows three traﬃc power control are also inherently coupled , astypes, namely, periodic data, intermittent data and the capacity available on each link depends on thelow frequency data through its contention-based transmission power. Moreover, speciﬁcally to multi-and contention-free channel access methods. In par- media transmissions, the application layer does notticular, for delay-sensitive applications, slots can be require full insulation from lower layers, but needsreserved every super-frame that allows contention- instead to perform source coding based on informa-free and high-priority access. The standard speciﬁes tion from the lower layers to maximize the multime-a reduced functionality device (RFD) and a full dia performance. Existing solutions often do notfunctionality device (FFD) in which only the latter provide adequate support for multimedia applica-can talk with other RFDs or FFDs and assume tions since the resource management, adaptation,the role of network coordinators/traﬃc forwarders. and protection strategies available in the lower lay-Distinguished on the basis of memory resource ers of the stack are optimized without explicitly con-availability and communication capability, this sidering the speciﬁc characteristics of multimediastandard introduces heterogeneity thus cutting applications. Similarly, multimedia compressiondeployment costs further. and streaming algorithms do not consider the mech- anisms provided by the lower layers for error pro-10.2.1. Open research issues tection and resource allocation . Though dedicated communication slots are pos- The additional challenges brought about by real-sible in Zigbee, the low data rate limits its applica- time streaming of multimedia content in WMSNsbility for multimedia applications. The standard call for new research on cross-layer optimizationdescribes a self-organizing network but heteroge- and cross-layer design methodologies, to leverage
952 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960potential improvements of exchanging information applications in WMSNs, by carefully leveraging andbetween diﬀerent layers of the communication controlling interactions among layers according tostack. However, the increased complexity of func- the applications requirements. Its design is basedtionalities needed to deliver QoS to multimedia on the following principles:applications needs to be managed as well. In partic-ular, it is important to keep some form of logical • Network layer QoS support enforced by a cross-separation of these functionalities to preserve layer controller. The proposed system providesupgradability and ease of design and testing. To this QoS support at the network layer, i.e., it providesaim, it is needed to specify standardized interfaces packet-level service diﬀerentiation in terms ofthat will allow leveraging these interactions. throughput, end-to-end packet error rate, and Although a consistent amount of recent papers delay. This is achieved by controlling operationshave focused on cross-layer design and improve- and interactions of functionalities at the physical,ment of protocols for WSNs, a systematic method- MAC, and network layers, based on a uniﬁedology to accurately model and leverage cross-layer logic that resides on a cross-layer controller thatinteractions is still largely missing. Most of the exist- manages resource allocation, adaptation, anding studies decompose the resource allocation prob- protection strategies based on the state oflem at diﬀerent layers, and consider allocation of the each functional block, as shown in Fig. 10. Theresources at each layer separately. In most cases, objective of the controller is to optimize someresource allocation problems are treated either heu- objective function, i.e., minimize energy con-ristically, or without considering cross-layer interde- sumption, while guaranteeing QoS requirementspendencies, or by considering pairwise interactions to application ﬂows. While all decisions arebetween isolated pairs of layers. jointly taken at the controller, implementation In , the cross-layer transmission of multime- of diﬀerent functionalities is kept separate fordia content over wireless networks is formalized as ease of design and upgradeability.an optimization problem. Several diﬀerent app- • UWB physical/MAC layer. The communicationroaches for cross-layer design of multimedia com- architecture is based on an integrated TH-IR-munications are discussed, including bottom-up UWB MAC and physical layer. Similarly toapproach, where the lower layers try to insulate CDMA, TH-IR-UWB allows several transmis-the higher layers from losses and channel capacity sions in parallel. Conversely, typical MACvariations, and top-down, where the higher layer protocols for sensor networks, such as con-protocols optimize their parameters at the next tention-based protocols based on CSMA/CA,lower layer. However, only single-hop networks require mutual temporal exclusion between neigh-are considered. boring transmitters. This allows devising MAC In , several techniques that provide signiﬁcant protocols with minimal coordination. Whileperformance gains through cross-layer optimizations CDMA usually entails complex transceivers andare surveyed. In particular, the improvements of cumbersome code assignment protocols, this isadaptive link layer techniques such as adaptive mod- FLOWulation and packet size optimization, joint allocation REQUIREMENTSof capacity and ﬂows (i.e., MAC and routing), joint CROSS LAYER UWBscheduling and rate allocation, are discussed. While Transceiver CONTROL UNIT GeoRouting (XLCU)still maintaining a strict layered architecture, it is Admission Next Hopshown how these cross-layer optimizations help Control Selection Capacity Channel Coder Source Coderimprove the spectral eﬃciency at the physical layer, Assignment Data Rate Coding QoSand the peak signal-to-noise ratio (PSNR) of the Rate Contractsvideo stream perceived by the user. Clearly, energy- QoS Schedulerconstrained multimedia sensors may need to leverage Rate Control NETWORKcross-layer interactions one step further. At the same COORDINATION UNIT (NCU)time, optimization metrics in the energy domain need LOCALIZATION TO NETWORKto be considered as well. SUBSYSTEM PEERS We are currently developing a new cross-layer SYNCHRONIZATION SUBSYSTEMcommunication architecture  whose objective isto reliably and ﬂexibly deliver QoS to heterogeneous Fig. 10. Cross-layer communication architecture.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 953 achievable with simple transceivers in TH-IR- into layer-speciﬁc considerations, there are also UWB. additional areas that need to be addressed. This• Receiver-centric scheduling for QoS traﬃc. One of section discusses the impact of recent advances in the major problems in multi-hop wireless environ- sensor-actuation, synchronization issues, spatial ments is that channel and interference vary with localization techniques, security and management the physical location of devices. For this reason, tools in the context of multimedia transmission. we believe that QoS provisioning should be based on receiver-centric scheduling of packets. This 12.1. Convergence of sensing and actuation way, the receiver can easily estimate the state of the medium at its side. Thus, it can optimally han- The challenges brought about by WMSNs are dle loss recovery and rate adaptation, thereby not to be limited to resource allocation problems. avoiding feedback overheads and latency, and In fact, WMSNs enable new application scenarios be responsive to the dynamics of the wireless link in synergy with other research areas. For example, using the information obtained locally. Distributed Robotics  has been a hot research• Dynamic channel coding. Adaptation to interfer- topic since the mid-1990s. In distributed robotics, ence at the receiver is achieved through dynamic a task is not completed by a single robot but by a channel coding, which can be seen as an alterna- team of collaborating robots. Information about tive form of power control, as it modulates the the surrounding environment is usually gathered energy per bit according to the interference per- by onboard sensors, and team members exchange ceived at the receiver. sensor information to move or perform actions• Geographical forwarding. We leverage UWB’s (e.g., collaborate to manipulate heavy objects). As positioning capabilities, to allow scalable geo- opposed to a single robot, a team of robots can per- graphical routing. The routing paths are selected ceive the environment from multiple disparate view- by the cross-layer controller by applying an points. In the recently proposed Wireless Sensor and admission control procedure that veriﬁes that Actor Networks (WSANs)  paradigm, the abil- each node on the path be able to provide the ity of the actors to perceive the environment can required service level. The required packet error be pushed one step further: a dense spatio-temporal rate and maximum allowed delay are calculated sampling of the environment, provided by a pre- at each step based on the relative advance of each deployed sensor network, can be exploited by the hop towards the destination. whole team of actors, thus increasing the ability of• Hop-by-hop QoS contracts. End-to-end QoS the team to accurately interact with the physical requirements are guaranteed by means of local environment. Furthermore, multimedia content decision. Each single device that participates in gathered by sensors can be used to provide the team the communication process is responsible for of actors with accurate vision from multiple per- locally guaranteeing given performance objec- spectives, while as of today collaborating actors tives. The global, end-to-end requirement is mostly rely on expensive onboard cameras. enforced by the joint local behaviors of the par- Coordination and communication algorithms for ticipating devices. static  and mobile  WSANs have been the• Multi-rate transmission. TH-IR-UWB allows focus of our research in recent years. In , we varying the data rate at the physical layer, by introduced a framework for communication and modifying the pulse repetition period. While this coordination problems in WSANs. The notions of functionality has not been fully explored so far, sensor–actor coordination and actor–actor coordi- it is possible to devise adaptive systems that mod- nation were introduced. The process of establishing ify the achievable data rate at the physical layer data paths between sensors and actors is referred to based on the perceived interference and on the as sensor–actor coordination. Once an event has required power consumption. been detected, actors coordinate to reconstruct it, to estimate its characteristics, and make a collabora- tive decision on how to perform the action. This12. Other research issues process is referred to as actor–actor coordination. In , we introduced a location management While most of the challenges in realizing a prac- scheme to handle the mobility of actors with mini-tical implementation of WMSNs can be classiﬁed mal energy consumption for resource-constrained
954 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960sensors. The proposed scheme is designed to reduce ments. In such cases, the ﬂow of two separate multi-the energy consumption with respect to existing media streams needs to be synchronized as the ﬁnallocalization services for ad hoc and sensor net- data available at the end user will comprise of bothworks. This is achieved through a combination of of these, played back under a joint timing con-location updating and location prediction, where straint. The task of coordinating such sequences isactors broadcast location updates limiting their called multimedia synchronization. The problemscope based on Voronoi diagrams, while sensors of multimedia synchronization is cross-layer in nat-predict the movement of actors based on Kalman ure and inﬂuences primarily the physical layer andﬁltering of previously received updates. the application layer. At the physical layer, data Clearly, further research is needed to fully lever- from diﬀerent media are multiplexed over sharedage the opportunities oﬀered by the integration of wireless connections, or are stored in common phys-actors and multimedia sensors in a wireless network. ical storage. The application layer is concerned with intermedia synchronization necessary for presenta- tion or playout, i.e., with the interactions between12.2. Network synchronization the multimedia application and the various media. The performance metrics for such inter-media Time synchronization is diﬃcult to achieve on a synchronization diﬀer from the cases in which onlynetwork-wide basis due to slow clock drift over voice or video streaming is desired. Synchronizationtime, eﬀect of temperature and humidity on clock can be applied to the playout of concurrent orfrequencies, coordination and correction amongst sequential streams of data, and also to the externalthousands of deployed nodes with low messaging events generated by a human user. In , it isoverhead, amongst others . The need for accu- argued that the average instantaneous delay varia-rate timing in WMSNs is stressed mainly for the fol- tion (skew) best measures inter-media synchroniza-lowing two scenarios: tion for continuous media. The maximum and minimum delay, used in typical real-time scheduling,• In-network processing schemes are often used in can be eﬀectively applied for discrete events associ- WMSNs in order to reduce traﬃc load in the net- ated with timed playout of text, graphics, and still work. However, ﬂows can only be aggregated at images. Minimizing the delay variation at each hop an intermediate node if the diﬀerence in the is a challenge yet unaddressed in the context of sen- packet generation times is within allowed sor networks, though the eﬀects of intermedia skew bounds. We believe that limited synchronization can be mitigated to an extent by dropping and dupli- could serve the requirements of WMSNs, in cating frames of the diﬀerent media streams so that which, it needs to be enforced only along the they play back in unison at the receiver. route chosen to the sink and for a speciﬁed length of time. Algorithms specially developed for sen- 12.4. Localization sor networks  can be easily adapted and large-scale coordination avoided. Determining the location of the sensor nodes• Real-time streaming needs clock synchronization with respect to a common reference, or in the con- at the sender and receiver to prevent buﬀer text of the object being monitored by them, is an underﬂows and overﬂows. Phase Locked Loops important aspect in multimedia applications. Cam- (PLL) help in maintaining clock frequencies at eras and microphones have limited ﬁeld of opera- the two ends but an eﬃcient digital implementa- tion and hence reachability and coverage are two tion of the Voltage Controlled Oscillator (VCO) important considerations that go into eﬃcient shar- is an important area of research. ing of monitoring tasks. Localization techniques help in allocating resources to events, deciding sens-12.3. Inter-media synchronization ing precision and ensuring complete monitoring of the area under study. Multimedia data returned by sensors can be het- In , this common space is provided by auto-erogeneous, and possibly correlated. As an example, matically determining the relative 3D positions ofvideo and audio samples could be collected over a audio sensors and actors through a closed formcommon geographical region, or still pictures may approximate solution based on time of ﬂight and timecomplement text data containing ﬁeld measure- diﬀerence of ﬂight. This approach however, requires
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 955that audio signals emitted by each sensor does not cations. Scalar or voice data may be rendered invis-interfere during the localization process which ible by embedding it in frames of video images thusimplies a network-wide ordering or code assignment. making eavesdropping diﬃcult. SensEye  uses a two-tier localization in whichcheap, low resolution cameras are used for object 12.6. Network managementdetection and higher resolution web-cameras areused for tracking. It uses two cameras with overlap- Multimedia network management, when appliedping coverage to localize an object and compute its to sensor networks, can be considered as a function-Cartesian (x, y, z) coordinates which, in turn, is used ality that encompasses resources, theories, tools andto intelligently wake up other nodes or to determine techniques to manipulate data provided by diﬀerentthe trajectory of the object. This work assumes that and possibly mixed media with the goal to extractthe orientations of the cameras are known relative relevant information . Taking into account theto a global reference frame and achieved through concerns of WMSNs, we believe that the design ofa GPS like or distributed technique. such systems should be inﬂuenced by the following factors:12.5. Network security • Reduced hardware/application requirement. Tools Security in WMSNs has recently caught the for WMSNs may comprise of hand-held devicesattention of the research community with increasing that are used by the network manager to conductapplications of sensors in military use. While the use on-site surveys. Light-weight application envi-of stronger codes, watermarking techniques, encryp- ronments like the Java Platform, Micro Editiontion algorithms, amongst others, have resulted in (Java ME)  or AJAX  could be used forsecured wireless communication, there are alto- local record manipulation as they considerablygether diﬀerent considerations in WMSNs. reduce traﬃc between the source and the distant As outlined in , a video sensor surveillance server (or sink). Java ME provides support forsystem may require in-network processing tech- networked and oﬄine applications that can beniques to reduce the amount of information ﬂowing downloaded dynamically. For distant web-basedin the network. At the aggregation point of the monitoring, AJAX builds upon XML and can beincoming streams, the packets would have to be used to perform simple, localized search andcompletely decoded and thus the computational modiﬁcation queries with short length messagescomplexity of the security algorithms must be low exchanged between the source and sink. Theseenough to allow real-time processing. There is hence tools may allow dynamic reassignment of goalsa trade-oﬀ between providing enhanced security to based on perceived QoS or events of interest.the data ﬂow by adopting a higher order code at • Independence of platform/programming environ-the source video sensor and permissible multimedia ment. Established proprietary tools like LabViewdelay requirements. Apart from devising eﬀective cannot be easily integrated with other languageslight-weight coding techniques, we believe that and there is a clear need of platform independent,eﬀorts in this area must be directed to leverage phys- general purpose monitoring tools. As a solution,ical layer strategies, as processing power on the bat- BeanWatcher , specially devised for WMSNs,tery-powered nodes is likely to be limited. can work with several languages, such as Java, The delta–sigma (DR) modulator for high-speed Java ME and C++. Besides, it provides somespeech processing is modiﬁed in  for simulta- visual components (e.g., thermometer, speedome-neously digitizing and authenticating sensor read- ter, gauge, and valued maps) to cover diﬀerentings. By exchanging simple keys, ﬁlter parameters types of sensory data and accommodates the factcan be decided that are used to encode the generated that diﬀerent streams from diﬀerent types ofstream, thus proving to be an computationally inex- applications need to be treated diﬀerently.pensive scheme. However, this technique has severalpractical diﬃculties including modulator matchingbetween the sender and receiver and precision track- 13. Conclusionsing of the signal for accurate demodulation. Otherareas that need to be explored are watermarking We discussed the state of the art of research onfor heterogeneous streams of voice and video appli- Wireless Multimedia Sensor Networks (WMSNs),
956 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960and outlined the main research challenges. Algo- Optical Instrumentation Engineers – Visual Communica-rithms, protocols, and hardware for the develop- tions and Image Processing, San Jose, CA, USA, January 2004.ment of WMSNs were surveyed, and open research  A. Aaron, S. Rane, R. Zhang, B. Girod, Wyner–Ziv codingissues discussed in detail. We classiﬁed currently for video: applications to compression and error resilience,oﬀ-the-shelf hardware as well as available research in: Proc. of IEEE Data Compression Conf. (DCC),prototypes for WMSNs. Furthermore, we discussed Snowbird, UT, March 2003, pp. 93–102.existing solutions and open research issues at the  A. Aaron, E. Setton, B. Girod, Towards practical Wyner– Ziv coding of video, in: Proc. of IEEE Intl. Conf. on Imageapplication, transport, network, link, and physical Processing (ICIP), Barcelona, Spain, September 2003.layers of the communication stack, along with possi-  R. Agre, L.P. Clare, G.J. Pottie, N.P. Romanov, Develop-ble cross-layer synergies and optimizations. We ment platform for self-organizing wireless sensor networks,pointed out how recent work undertaken in in: Proc. of Society of Photo-Optical InstrumentationWyner–Ziv coding at the application layer, special- Engineers – Aerosense, Orlando, FL, USA, March 1999.  O. Akan, I.F. Akyildiz, Event-to-sink reliable transport inized spatio-temporal transport layer solutions, delay wireless sensor networks, IEEE/ACM Trans. Network 13bounded routing, multi-channel MAC protocols, (5) (2005) 1003–1017.and UWB technology, amongst others, seem most  K. Akkaya, M. Younis, An energy-aware QoS routingpromising research directions in developing practical protocol for wireless sensor networks, in: Proc. of Intl.WMSNs. We believe that this research area will Conf. on Distributed Computing Systems Workshops (ICSDSW), Washington, DC, 2003.attract the attention of many researchers and that  K. Akkaya, M. Younis, A survey of routing protocols init will push one step further our ability to observe wireless sensor networks, Ad Hoc Network (Elsevier) 3 (3)the physical environment and interact with it. (2005) 325–349.  I.F. Akyildiz, I.H. Kasimoglu, Wireless sensor and actor networks: Research challenges, Ad Hoc Networks (Else-Acknowledgement vier) 2 (4) (2004) 351–367.  I.F. Akyildiz, W.-Y. Lee, M.C. Vuran, S. Mohanty, Next The authors would like to thank Vehbi C. Gun- generation/dynamic spectrum access/cognitive radio wire-gor, Dario Pompili, and Mehmet C. Vuran for their less networks: a survey, Comput. Networks (Elsevier) 50valuable comments. (13) (2006) 2127–2159.  I.F. Akyildiz, W. Su, Y. Sankarasubramaniam, E. Cayirci, Wireless sensor networks: a survey, Comput. NetworksReferences (Elsevier) 38 (4) (2002) 393–422.  J.N. Al-Karaki, A.E. Kamal, Routing techniques in wire-  Acroname GARCIA Robotic Platform. <http://www.acro- less sensor networks: a survey, IEEE Wireless Commun. 11 name.com/garcia/garcia.html>. (6) (2004) 6–28.  Advanced Video Coding for Generic Audiovisual Services,  M.I. Alghamdi, T. Xie, X. Qin, PARM: A power-aware ITU-T Recommendation H.264. message scheduling algorithm for real-time wireless net-  Crossbow. <http://www.xbow.com>. works, in: Proc. of ACM Workshop on Wireless Multi-  Crossbow MICA2 Mote Speciﬁcations. <http://www. media Networking and Performance Modeling (WMuNeP), xbow.com>. Montreal, Que., Canada, 2005, pp. 86–92.  Crossbow MICAz Mote Speciﬁcations. <http://www.xbow.  Anuj Batra et al., Multi-band OFDM physical layer com>. proposal for IEEE 802.15 Task Group 3a, IEEE P802.15  Crossbow TelosB Mote Speciﬁcations. <http://www.xbow. Working Group for Wireless Personal Area Networks com>. (WPANs), March 2004.  IEEE 802.15 wpan task group 4 (tg4). <http://group-  R. Biswas, K.R. Chowdhury, D.P. Agrawal, Optimal data- er.ieee.org/groups/802/15/pub/TG4.html>. centric attribute allocation and retrieval (DCAAR) scheme  Java Platform, Micro Edition. <http://java.sun.com/jav- for wireless sensor networks, in: Proc. of IEEE Intl. Conf. ame/index.jsp>. on Mobile Ad-hoc and Sensor Systems (MASS), Washing-  Revision of Part 15 of the Commission’s Rules Regarding ton, DC, USA, November 2005. Ultra-Wideband Transmission Systems. First note and  J.-Y.L. Boudec, P. Thiran, Network Calculus, LNCS, vol. Order, Federal Communications Commission, ET-Docket 2050, Springer-Verlag, New York, 2001. 98-153, adopted February 14, 2002, released April 22, 2002.  A. Boulis, M. Srivastava, Node-level energy management The Stargate Platform. <http://www.xbow.com/Products/ for sensor networks in the presence of multiple applications, Xscale.htm>. in: Proc. of IEEE Intl. Conf. on Pervasive Computing and Video Coding for Low Bit Rate Communication, ITU-T Communications (PerCom), Dallas – Forth Worth, TX, Recommendation H.263. USA, 2003, pp. 41–49. JPEG2000 Requirements and Proﬁles, ISO/IEC JTC1/  J. Campbell, P.B. Gibbons, S. Nath, P. Pillai, S. Seshan, SC29/WG1 N1271, March 1999. R. Sukthankar, IrisNet: an Internet-scale architecture for A. Aaron, S. Rane, E. Setton, B. Girod, Transform-domain multimedia sensors, in: Proc. of the ACM Multimedia Wyner–Ziv codec for video, in: Proc. of Society of Photo- Conference, 2005.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 957 Y.U. Cao, A.S. Fukunaga, A.B. Kahng, Cooperative  W. Feng, B. Code, E. Kaiser, M. Shea, W. Feng, L. Bavoil, mobile robotics: antecedents and directions, Autonom. Panoptes: scalable low-power video sensor networking Robots 4 (1997) 1–23. technologies, in: Proc. of ACM Multimedia, Berkeley, J. Capone, I. Stavrakakis, Delivering QoS requirements to CA, USA, November 2003. traﬃc with diverse delay tolerances in a TDMA environ-  W. Feng, N.B.W. Feng, Dissecting the video sensing ment, IEEE/ACM Trans. Network. 7 (1) (1999) 75–87. landscape, in: Proc. of the ACM Workshop on Network M. Chiang, Balancing transport and physical layers in and Operating System Support for Digital Audio and Video wireless multihop networks: jointly optimal congestion (NOSSDAV), Stevenson, WA, USA, June 2005. control and power control, IEEE J. Select. Areas Commun.  D. Gay, P. Levis, R. von Behren, M. Welsh, E. Brewer, 23 (1) (2005) 104–116. D. Culler, The nesC language: a holistic approach to net- K.R. Chowdhury, N. Nandiraju, D. Cavalcanti, D.P. work embedded systems, in: Proc. of the ACM SIGPLAN Agrawal, CMAC-A multi-channel energy eﬃcient MAC 2003 Conf. on Programming Language Design and Imple- for wireless sensor networks, in: Proc. of IEEE Wireless mentation (PLDI), San Diego, CA, USA, June 2003. Communications and Networking Conference (WCNC),  S. Gezici, Z. Tian, G.B. Giannakis, H. Kobayashi, A.F. Las Vegas, NV, April 2006. Molisch, H.V. Poor, Z. Sahinoglu, Localization via ultra- D. Chu, K. Lin, A. Linares, G. Nguyen, J.M. Hellerstein, wideband radios, IEEE Signal Process. Mag. 22 (4) (2005) sdlib: a sensor network data and communications library 70–84. for rapid and robust application development, in: Proc. of  B. Girod, A. Aaron, S. Rane, D. Rebollo-Monedero, Information Processing in Sensor Networks (IPSN), Nash- Distributed video coding, Proc. IEEE 93 (1) (2005) 71–83. ville, TN, USA, April 2006.  B. Girod, M. Kalman, Y. Liang, R. Zhang, Advances in M. Chu, J.E. Reich, F. Zhao, Distributed attention for channel-adaptive video streaming, Wireless Commun. large video sensor networks, in: Proc. of the Institute of Mobile Comput. 2 (6) (2002) 549–552. Defence and Strategic Studies (IDSS), London, UK,  V. Gungor, O. Akan, I. Akyildiz, A real-time and reliable February 2004. transport protocol for wireless sensor and actor networks, R. Cruz, A calculus for network delay. I. Network elements submitted for publication. in isolation, IEEE Trans. Informat. Theory 37 (1) (1991)  C. Guo, L.C. Zhong, J.M. Rabaey, Low power distributed 114–131. MAC for Ad Hoc sensor radio networks, in: Proc. of the R. Cucchiara, Multimedia surveillance systems, in: Proc. of IEEE Global Communications Conference (GLOBE- ACM Intl. Workshop on Video Surveillance and Sensor COM), San Antonio, TX, November 2001. Networks, Singapore, November 2005.  E. Gurses, O.B. Akan, Multimedia communication in S. Cui, R. Madan, A. Goldsmith, S. Lall, Energy-delay wireless sensor networks, Ann. Telecommun. 60 (7–8) tradeoﬀs for data collection in TDMA-based sensor (2005) 799–827. networks, in: Proc. of IEEE Intl. Conf. on Communications  S. Haykin, Cognitive radio: Brain-empowered wireless (ICC), Seoul, Korea, May 2005, pp. 3278–3284. communications, IEEE J. Select. Areas Commun. 23 (2) F. Cuomo, C. Martello, A. Baiocchi, F. Capriotti, Radio (2005) 201–220. resource sharing for ad-hoc networking with UWB, IEEE J.  T. He, J.A. Stankovic, C. Lu, T.F. Abdelzaher, A spatio- Select. Areas Commun. 20 (9) (2002) 1722–1732. temporal communication protocol for wireless sensor T.V. Dam, K. Langendoen, An adaptive energy-eﬃcient networks, IEEE Trans. Parallel Distr. Syst. 16 (10) (2005) MAC protocol for wireless sensor networks, in: Proc. of the 995–1006. ACM Conf. on Embedded Networked Sensor Systems  T. Herman, S. Tixeuil, A distributed TDMA slot assign- (SenSys), Los Angeles, CA, USA, November 2003. ment algorithm for wireless sensor networks, in: Proc. of I. Downes, L.B. Rad, H. Aghajan, Development of a mote the Workshop on Algorithmic Aspects of Wireless Sensor for wireless image sensor networks, in: Proc. of COGnitive Networks (ALGOSENSORS), Finland, 2004. systems with Interactive Sensors (COGIS), Paris, France,  R. Holman, J. Stanley, T. Ozkan-Haller, Applying video March 2006. sensor networks to nearshore environment monitoring, A. Dunkels, T. Voigt, J. Alonso, Making TCP/IP viable for IEEE Perv. Comput. 2 (4) (2003) 14–21. wireless sensor networks, in: Proc. of European Workshop  F. Hu, S. Kumar, Multimedia query with QoS consider- on Wireless Sensor Networks (EWSN), Berlin, Germany, ations for wireless sensor networks in telemedicine, in: Proc. January 2004. of Society of Photo-Optical Instrumentation Engineers – A. Dunkels, T. Voigt, J. Alonso, H. Ritter, Distributed Intl. Conf. on Internet Multimedia Management Systems, TCP caching for wireless sensor networks, in: Proc. of the Orlando, FL, September 2003. Mediterranean Ad Hoc Networking Workshop (MedHoc-  Y.G. Iyer, S. Gandham, S. Venkatesan, STCP: a Generic Net), June 2004. Transport Layer Protocol for Wireless Sensor Networks, in: Y. Eisenberg, C.E. Luna, T.N. Pappas, R. Berry, A.K. Proc. of IEEE Intl. Conf. on Computer Communications Katsaggelos, Joint source coding and transmission power and Networks (ICCCN), San Diego, CA, USA, October management for energy eﬃcient wireless video communi- 2005, pp. 449–454. cations, IEEE Trans. Circ. Syst. Video Technol. 12 (6)  D. James, L. Klibanov, G. Tompkins, S.J. Dixon-Warren, (2002) 411–424. Inside CMOS Image Sensor Technology. Chipworks White E. Felemban, C.-G. Lee, E. Ekici, MMSPEED: Multipath Paper. <http://www.chipworks.com/resources/whitepapers/ multi-SPEED protocol for QoS guarantee of reliability and Inside-CMOS.pdf>. timeliness in wireless sensor networks, IEEE Trans. Mobile  S.K. Jayaweera, An energy-eﬃcient virtual MIMO com- Comput. 5 (6) (2006) 738–754. munications architecture based on V-BLAST processing for
958 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 distributed wireless sensor networks, in: Proc. of IEEE on Systems, Man and Cybernetics, October 2004, pp. 6081– Intl. Conf. on Sensor and Ad-hoc Communications and 6086. Networks (SECON), Santa Clara, CA, USA, October  S. Mao, D. Bushmitch, S. Narayanan, S.S. Panwar, MRTP: 2004. a multiﬂow real-time transport protocol for Ad Hoc S.K. Jayaweera, M.L. Chebolu, Virtual MIMO and networks, IEEE Trans. Multimedia 8 (2) (2006) 356–369. distributed signal processing for sensor networks – an  C.B. Margi, V. Petkov, K. Obraczka, R. Manduchi, integrated approach, in: Proc. of IEEE Intl. Conf. on Characterizing energy consumption in a visual sensor Communications (ICC), Seoul, Korea, May 2005. network testbed, in: Proc. of IEEE/Create-Net Intl. Conf. X. Jiang, J. Polastre, D. Culler, Perpetual environmentally on Testbeds and Research Infrastructures for the Develop- powered sensor networks, in: Proc. of IEEE Workshop on ment of Networks and Communities (TridentCom), Barce- Sensor Platform, Tools and Design Methods for Net- lona, Spain, March 2006. worked Embedded Systems (SPOTS), Los Angeles, CA,  D. McIntire, Energy Beneﬁts of 32-bit Microprocessor USA, April 2005. Wireless Sensing Systems, Sensoria Corporation White S. Kim, R. Fonseca, D. Culler, Reliable transfer in wireless Paper. sensor networks, in: Proc. of IEEE Intl. Conf. on Sensor  L. Meier, P. Blum, L. Thiele, Interval synchronization of and Ad-hoc Communications and Networks (SECON), drift-constraint clocks in Ad-Hoc sensor networks, in: Proc. Santa Clara, CA, October 2004. of ACM Intl. Symposium on Mobile Ad Hoc Networking J. Koshy, R. Pandey, VM*: Synthesizing scalable runtime and Computing (MobiHoc), Tokyo, Japan, May 2004. environments for sensor networks, in: Proc. of the ACM  T. Melodia, I.F. Akyildiz, Cross-layer QoS support for Conf. on Embedded Networked Sensor Systems (SenSys), UWB wireless multimedia sensor networks, in preparation. San Diego, CA, November 2005.  T. Melodia, D. Pompili, I.F. Akyildiz, On the interdepen- K. Kredo, P. Mohapatra, Medium access control in dence of distributed topology control and geographical wireless sensor networks, Comput. Networks (Elsevier), in routing in ad hoc and sensor networks, IEEE J. Select. press, doi:10.1016/j.comnet.2006.06.012. Areas Commun. 23 (3) (2005) 520–532. P. Kulkarni, D. Ganesan, P. Shenoy, The case for multi-tier  T. Melodia, D. Pompili, I.F. Akyildiz, A communication camera sensor network, in: Proc. of the ACM Workshop on architecture for mobile wireless sensor and actor networks, Network and Operating System Support for Digital Audio in: Proc. of IEEE Intl. Conf. on Sensor and Ad-hoc and Video (NOSSDAV), Stevenson, WA, USA, June 2005. Communications and Networks (SECON), Reston, VA, P. Kulkarni, D. Ganesan, P. Shenoy, Q. Lu, SensEye: a USA, September 2006. multi-tier camera sensor network, in: Proc. of ACM  T. Melodia, D. Pompili, V.C. Gungor, I.F. Akyildiz, A Multimedia, Singapore, November 2005. distributed coordination framework for wireless sensor and S.S. Kulkarni, M. Arumugam, TDMA service for sensor actor networks, in: Proc. of ACM Intl. Symposium on networks, in: Proc. of the Intl. Conf. on Distributed Mobile Ad Hoc Networking and Computing (MobiHoc), Computing Systems Workshops (ICDCSW), Washington, Urbana-Champaign, IL, May 2005. DC, USA, 2004, pp. 604–609.  R. Merz, J. Widmer, J.-Y.L. Boudec, B. Radunovic, A joint D. Kundur, T. Zourntos, N.J. Mathai, Lightweight security PHY/MAC architecture for low-radiated power TH-UWB principles for distributed multimedia based sensor net- wireless ad-hoc networks, Wireless Commun. Mobile works, in: Proc. of Asilomar Conf. on Signals, Systems and Comput. J. 5 (5) (2005) 567–580. Computers, November 2004, pp. 368–372.  M.J. Miller, N.H. Vaidya, A MAC protocol to reduce P. Kyasanur, N.H. Vaidya, Capacity of multi-channel sensor network energy consumption using a wakeup radio, wireless networks: Impact of number of channels and IEEE Trans. Mobile Comput. 4 (3) (2005) 228–242. interfaces, in: Proc. of ACM Intl. Conf. on Mobile  A. Mishra, S. Banerjee, W. Arbaugh, Weighted coloring Computing and Networking (MobiCom), Cologne, based channel assignment for WLANs, Mobile Comput. Germany, August 2005. Commun. Rev. 9 (3) (2005) 19–31. Q. Li, D. Rus, Global clock synchronization in sensor  S. Misra, M. Reisslein, G. Xue, A survey of multimedia networks, IEEE Trans. Comput. 55 (2) (2006) 214–226. streaming in wireless sensor networks, submitted for A. Lins, E.F. Nakamura, A.A.F. Loureiro, C.J.N. Coelho publication. Jr., BeanWatcher: a tool to generate multimedia monitoring  P.D. Mitcheson, T.C. Green, E.M. Yeatman, A.S. Holmes, applications for wireless sensor networks, in: Proc. of IFIP/ Architectures for vibration-driven micropower generators, IEEE Intl. Conf. on Management of Multimedia Networks J. Microelectromech. Syst. 13 (3) (2004) 429–440. and Services, 2003, pp. 128–141.  M. Miyabayashi, N. Wakamiya, M. Murata, H. Miyahara, T.D.C. Little, A framework for synchronous delivery of MPEG-TFRCP: Video transfer with TCP-friendly rate time-dependent multimedia data, J. Multimedia Syst. 1 (2) control protocol, in: Proc. of IEEE Intl. Conf. on (1993) 87–94. Communications (ICC), Helsinki, June 2001. S. Lu, V. Bharghavan, R. Srikant, Fair scheduling in  S. Nath, Y. Ke, P.B. Gibbons, B. Karp, S. Seshan, A wireless packet networks, IEEE/ACM Trans. Network. 7 distributed ﬁltering architecture for multimedia sensors, (4) (1999) 473–489. Intel Research Technical Report IRP-TR-04-16, August X. Lu, E. Erkip, Y. Wang, D. Goodman, Power eﬃcient 2004. multimedia communication over wireless channels, IEEE J.  T. Nguyen, S. Cheung, Multimedia streaming with multiple Select. Areas Commun. 21 (10) (2003) 1738–1751. TCP connections, in: Proc. of the IEEE Intl. Performance X. Luo, K. Zheng, Y. Pan, Z. Wu, A TCP/IP implemen- Computing and Communications Conference (IPCCC), tation for wireless sensor networks, in: Proc. of IEEE Conf. Arizona, April 2005.
I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 959  E. Ould-Ahmed-Vall, D. Blough, B.S. Heck, G.F. Riley, Sivalingam (Eds.), Wireless Sensor Networks, Springer, Distributed global identiﬁcation for sensor networks, in: New York, 2005. Proc. of IEEE Intl. Conf. on Mobile Ad-hoc and Sen-  M.V.D. Schaar, S. Shankar, Cross-layer wireless multi- sor Systems (MASS), Washington, DC, USA, November media transmission: Challenges, principles and new para- 2005. digms, IEEE Wireless Commun. 12 (4) (2005) 50–58.  J. Paradiso, T. Starner, Energy scavenging for mobile and  R.A. Scholtz, N.Z. Win, Impulse radio: How it works, wireless electronics, IEEE Perv. Comput. 4 (1) (2005) 18– IEEE Commun. Lett. 2 (2) (1998) 36–38. 27.  C. Schurgers, V. Tsiatsis, S. Ganeriwal, M. Srivastava,  M. Perillo, W. Heinzelman, Sensor management policies to Optimizing sensor networks in the energy-latency-density provide application QoS, Ad Hoc Networks (Elsevier) 1 (2– design space, IEEE Trans. Mobile Comput. 1 (1) (2002) 70– 3) (2003) 235–246. 80.  B.W. Perry, Ajax Hacks, O’Reilly Publications, 2006.  C. Schurgers, V. Tsiatsis, S. Ganeriwal, M. Srivastava,  G.J. Pottie, W.J. Kaiser, Wireless integrated network Topology management for sensor networks: Exploiting sensors, Commun. ACM 43 (May) (2000) 51–58. latency and density, in: Proc. of ACM Intl. Conf. on R. Puri, K. Ramchandran, PRISM: a new robust video Mobile Computing and Networking (MobiCom), Atlanta, coding architecture based on distributed compression GA, USA, September 2002. principles, in: Proc. of the Allerton Conf. on Communica-  E. Setton, T. Yoo, X. Zhu, A. Goldsmith, B. Girod, Cross- tion, Control, and Computing, Allerton, IL, USA, October layer design of ad hoc networks for real-time video 2002. streaming, IEEE Wireless Commun. 12 (4) (2005) 59–65. V. Raghunathan, S. Ganeriwal, M. Srivastava, C. Schur-  D. Slepian, J. Wolf, Noiseless coding of correlated infor- gers, Energy eﬃcient wireless packet scheduling and fair mation sources, IEEE Trans. Informat. Theory 19 (4) queuing, ACM Trans. Embedded Comput. Syst. 3 (1) (1973) 471–480. (2004) 3–23.  S. Soro, W.B. Heinzelman, On the coverage problem in M. Rahimi, S. Ahmadian, D. Zats, J. Garcia, M. Srivast- video-based wireless sensor networks, in: Proc. of the IEEE ava, D. Estrin, Deep vision: Experiments in exploiting Intl. Conf. on Broadband Communications, Networks, and vision in wireless sensor networks, submitted for Systems (BroadNets), Boston, MA, USA, October 2005. publication.  F. Stann, J. Heidemann, RMST: Reliable data transport in M. Rahimi, R. Baer, O. Iroezi, J. Garcia, J. Warrior, sensor networks, in: Proc. of IEEE Sensor Network D. Estrin, M. Srivastava, Cyclops: in situ image sensing and Protocols and Applications (SNPA), Anchorage, Alaska, interpretation in wireless sensor networks, in: Proc. of the USA, April 2003, pp. 102–112. ACM Conf. on Embedded Networked Sensor Systems  H. Stockdon, R. Holman, Estimation of wave phase speed (SenSys), San Diego, CA, November 2005. and nearshore bathymetry from video imagery, J. Geophys. V.C. Raykar, I. Kozintsev, R. Lienhart, Position calibra- Res. 105 (C9) (2000) 22,015–22,033. tion of microphones and loudspeakers in distributed  B. Sundararaman, U. Buy, A. Kshemkalyani, Clock computing platforms, IEEE Trans. Speech Audio Process. synchronization for wireless sensor networks: a survey, 13 (1) (2005) 70–83. Ad Hoc Networks (Elsevier) 3 (3) (2005) 281–323. J. Reed, Introduction to Ultra Wideband Communication  K. Sundaresan, V. Anantharaman, H.-Y. Hsieh, R. Systems, Prentice Hall, Englewood Cliﬀs, NJ, 2005. Sivakumar, ATP: a reliable transport protocol for Ad A.A. Reeves, Remote Monitoring of patients suﬀering from Hoc networks, IEEE Trans. Mobile Comput. 4 (6) (2005) early symptoms of Dementia, in: Intl. Workshop on 588–603. Wearable and Implantable Body Sensor Networks, Lon-  J. Tourrilhes, Packet frame grouping: Improving IP multi- don, UK, April 2005. media performance over CSMA/CA, in: Proc. of IEEE A. Rowe, C. Rosenberg, I. Nourbakhsh, A low cost Conf. on Universal Personal Communications, Florence, embedded color vision system, in: Proc. of the IEEE/RSJ Italy, October 1998. Intl. Conf. on Intelligent Robots and Systems (IROS),  T. Vu, D. Reschke, W. Horn, Dynamic packet size Lausanne, Switzerland, October 2002. mechanism (DPSM) for multimedia in wireless networks, Y. Sankarasubramaniam, I.F. Akyildiz, S.W. McLaughlin, in: Proc. of Multimediale Informations- und Kommunika- Energy eﬃciency based packet size optimization in wireless tions systeme (MIK), Erfurt, Germany, September 2002. sensor networks, in: Proc. of IEEE Sensor Network  M.C. Vuran, O.B. Akan, I.F. Akyildiz, Spatio-temporal Protocols and Applications (SNPA), Anchorage, Alaska, correlation: Theory and applications for wireless sensor USA, April 2003, pp. 1–8. networks, Comput. Networks (Elsevier) 45 (3) (2004) 245– C. Santivanez, I. Stavrakakis, Study of various TDMA 259. schemes for wireless networks in the presence of deadlines  M.C. Vuran, I.F. Akyildiz, Cross-layer analysis of error and overhead, IEEE J. Select. Areas Commun. 17 (7) (1999) control in wireless sensor networks, in: Proc. of IEEE Intl. 1284–1304. Conf. on Sensor and Ad-hoc Communications and Net- L. Savidge, H. Lee, H. Aghajan, A. Goldsmith, QoS-based works (SECON), Reston, VA, USA, September 2006. geographic routing for event-driven image sensor networks,  C.Y. Wan, A.T. Campbell, L. Krishnamurthy, PSFQ: a in: Proc. of IEEE/CreateNet Intl. Workshop on Broadband reliable transport protocol for wireless sensor networks, in: Advanced Sensor Networks (BaseNets), Boston, MA, Proc. of ACM Workshop on Wireless Sensor Networks and October 2005. Applications (WSNA), Atlanta, GA, September 2002. A. Savvides, L. Girod, M. Srivastava, D. Estrin, Localiza-  C.Y. Wan, S.B. Eisenman, A.T. Campbell, CODA: Con- tion in sensor networks, in: T.Z.C.S. Raghavendra, K.M. gestion detection and avoidance in sensor networks, in:
960 I.F. Akyildiz et al. / Computer Networks 51 (2007) 921–960 Proc. of the ACM Conf. on Embedded Networked Sensor Award (IEEE Communications Society) for his paper entitled Systems (SenSys), Los Angeles, CA, November 2003. ‘‘Multimedia Group Synchronization Protocols for Integrated M.Z. Win, R.A. Scholtz, Ultra-wide bandwidth time- Services Architectures’’ published in the IEEE Journal of Selected hopping spread-spectrum impulse radio for wireless multi- Areas in Communications (JSAC) in January 1996. He received the ple-access communication, IEEE Trans. Commun. 48 (4) 2002 IEEE Harry M. Goode Memorial Award (IEEE Computer (2000) 679–689. Society) with the citation ‘‘for signiﬁcant and pioneering contri- A. Wyner, J. Ziv, The rate-distortion function for source butions to advanced architectures and protocols for wireless and coding with side information at the decoder, IEEE Trans. satellite networking’’. He received the 2003 IEEE Best Tutorial Informat. Theory 22 (January) (1976) 1–10. Award (IEEE Communication Society) for his paper entitled ‘‘A Z. Xiong, A.D. Liveris, S. Cheng, Distributed source Survey on Sensor Networks,’’ published in IEEE Communications coding for sensor networks, IEEE Signal Process. Mag. Magazine, in August 2002. He also received the 2003 ACM Sig- 21 (September) (2004) 80–94. mobile Outstanding Contribution Award with the citation ‘‘for L. Yang, G.B. Giannakis, Ultra-wideband communica- pioneering contributions in the area of mobility and resource tions: an idea whose time has come, IEEE Signal Process. management for wireless communication networks’’. He received Mag. 3 (6) (2004) 26–54. the 2004 Georgia Tech Faculty Research Author Award for his W. Ye, J. Heidemann, D. Estrin, Medium access control ‘‘outstanding record of publications of papers between 1999– with coordinated, adaptive sleeping for wireless sensor 2003’’. He also received the 2005 Distinguished Faculty Achieve- networks, IEEE Trans. Network. 12 (3) (2004) 493–506. ment Award from School of ECE, Georgia Tech. He has been a K. Yeung, 802.11a Modeling and MAC enhancements for Fellow of the Association for Computing Machinery (ACM) and high speed rate adaptive networks, Technical Report IEEE Fellow since 1996. UCLA, 2002. W. Yu, Z. Sahinoglu, A. Vetro, Energy-eﬃcient JPEG 2000 image transmission over wireless sensor networks, in: Proc. Tommaso Melodia received his Laurea of IEEE Global Communications Conference (GLOBE- and Doctorate degrees in Telecommuni- COM), Dallas, TX, USA, January 2004, pp. 2738–2743. cations Engineering from the University X. Zhu, B. Girod, Distributed rate allocation for multi- of Rome ‘‘La Sapienza’’, Rome, Italy, in stream video transmission over ad hoc networks, in: Proc. 2001 and 2005, respectively. He is cur- of IEEE Intl. Conf. on Image Processing (ICIP), Genoa, rently pursuing his Ph.D. in Electrical Italy, September 2005, pp. 157–160. and Computer Engineering while ´ B. Zitova, J. Flusser, Image registration methods: a survey, working as a research assistant at the Image Vis. Comput. 21 (11) (2003) 977–1000. Broadband and Wireless Networking M. Zuniga, B. Krishnamachari, Integrating future large- Laboratory (BWN-Lab), Georgia Insti- scale sensor networks with the Internet, USC Computer tute of Technology, Atlanta, under the Science Technical Report CS 03-792, 2003. guidance of Dr. Ian F. Akyildiz. His main research interests are in wireless sensor and actor networks, wireless multimedia sensor networks, underwater acoustic sensor networks, and in wireless Ian F. Akyildiz received the B.S., M.S., networking in general. He is the recipient of the BWN-Lab and Ph.D. degrees in Computer Engi- Researcher of the Year 2004 award. neering from the University of Erlangen- Nuernberg, Germany, in 1978, 1981 and 1984, respectively. Kaushik R. Chowdhury received his B.E. Currently, he is the Ken Byers Distin- degree in Electronics Engineering with guished Chair Professor with the School distinction from VJTI, Mumbai Univer- of Electrical and Computer Engineering, sity, India, in 2003. He received his M.S. Georgia Institute of Technology, Atlanta, degree in computer science from the and Director of Broadband and Wireless University of Cincinnati, OH, in 2006, Networking Laboratory. He is an Editor- graduating with the best thesis award.in-Chief of Computer Networks Journal (Elsevier) as well as the He is currently a Research Assistant infounding Editor-in-Chief of the Ad Hoc Network Journal (Else- the Broadband and Wireless Networkingvier). His current research interests are in next generation wireless Laboratory and pursuing his Ph.D.networks, sensor networks, and wireless mesh networks. degree at the School of Electrical and He received the ‘‘Don Federico Santa Maria Medal’’ for his Computer Engineering, Georgia Institute of Technology,services to the Universidad of Federico Santa Maria, in 1986. From Atlanta, GA. His current research interests include multichannel1989 to 1998, he served as a National Lecturer for ACM and medium access protocols, dynamic spectrum management, andreceived the ACM Outstanding Distinguished Lecturer Award in resource allocation in wireless multimedia sensor networks. He is1994. He received the 1997 IEEE Leonard G. Abraham Prize a student member of IEEE.