Thesis presentation Slides for Doctorale deplomat obtention of Ph.D. Aliouat Ahcen. Defended on 31-05-2023 in LASA Laboratory, Electronics Department, Faculty of Technology, Badji Mokhtar - Annaba University. Algeria
The presentation title is: Study and Implementation of an Object-based Video Encoder for Embedded Wireless Video Surveillance Systems.
Research Question: How can we detect ROI in a captured video to ensure high-quality encoding and transmission over a WMSN while minimizing bitrate and energy consumption?
Slide for Multi Object Tracking by Md. Minhazul Haque, Rajshahi University of Engineering and Technology
* Object
* Object Tracking
* Application
* Background Study
* How it works
* Multi-Object Tracking
* Solution
* Future Works
Image Compression: It is the Art & Science of reducing the amount of data required to represent an image
The number of images compressed and decompressed daily is innumerable
Slide for Multi Object Tracking by Md. Minhazul Haque, Rajshahi University of Engineering and Technology
* Object
* Object Tracking
* Application
* Background Study
* How it works
* Multi-Object Tracking
* Solution
* Future Works
Image Compression: It is the Art & Science of reducing the amount of data required to represent an image
The number of images compressed and decompressed daily is innumerable
A fast-paced introduction to Deep Learning that starts with a simple yet complete neural network (no frameworks), followed by an overview of activation functions, cost functions, backpropagation, and then a quick dive into CNNs. Next we'll create a neural network using Keras, followed by an introduction to TensorFlow and TensorBoard. For best results, familiarity with basic vectors and matrices, inner (aka "dot") products of vectors, and rudimentary Python is definitely helpful.
Computer Vision: Correlation, Convolution, and GradientAhmed Gad
Three important operations in computer vision are explained starting with each one got explained and implemented in Python.
Generally, all of these three operations have many similarities in as they follow the same general steps but there are some subtle changes. The main change is using different masks.
A presentation on Human Activity Recognition catered to the audience from an HCI or CS background. (Based on research by Bulling, A. et al. 2014. A tutorial on human activity recognition using body-worn inertial sensors. CSUR. 46, 3 (2014), 33.)
Single Image Super-Resolution from Transformed Self-Exemplars (CVPR 2015)Jia-Bin Huang
Self-similarity based super-resolution (SR) algorithms are able to produce visually pleasing results without extensive training on external databases. Such algorithms exploit the statistical prior that patches in a natural image tend to recur within and across scales of the same image. However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene. In this paper, we extend self-similarity based SR to overcome this drawback. We expand the internal patch search space by allowing geometric variations. We do so by explicitly localizing planes in the scene and using the detected perspective geometry to guide the patch search process. We also incorporate additional affine transformations to accommodate local shape variations. We propose a compositional model to simultaneously handle both types of transformations. We extensively evaluate the performance in both urban and natural scenes. Even without using any external training databases, we achieve significantly superior results on urban scenes, while maintaining comparable performance on natural scenes as other state-of-the-art SR algorithms.
http://bit.ly/selfexemplarsr
발표자: 이인웅 (연세대 박사과정)
발표일: 2017.12.
개요:
영상에서 사람의 행동을 인식하는 방법은 크게 영상에서 직접적으로 행동 라벨을 추출하는 것과 자세 정보를 기반으로 행동 라벨을 추출하는 것으로 나뉠 수 있습니다.
본 발표는 행동 인식에 대한 전반적인 개요를 설명하고 그 중에서도 사람의 자세 정보를 기반으로 하는 행동 인식 기술에 초점을 두고 최근 ICCV 2017 학회에서 발표된 Temporal Sliding LSTM 네트워크를 활용한 행동 인식 기술을 중점적으로 설명합니다. 구체적으로, 스켈레톤 기반 행동 인식 이슈, 제안하는 방법과 실험 결과들이 소개되고 앞으로 나아갈 만한 새로운 연구 이슈들도 추가적으로 설명합니다.
Ukrainian Catholic University
Faculty of Applied Sciences
Data Science Master Program
January 22nd
Abstract. The thesis introduces the reader to the concepts of edge computing in terms of person re-identification and tracking problem. It describes the challenges, limitations, and current state-of-the-art solutions. The author proposed a pipeline for the task, launched several experiments on validating different parts of the system, and provided a theoretical explanation of the person re-identification process in the overlapping multi-camera environment.
Intel RealSense D435 3D Active IR Stereo Depth Camera 2018 teardown reverse c...system_plus
The 3D camera is using infrared active stereo depth and red/green/blue sensors, and a VCSEL projector.
More information on that report at http://www.systemplus.fr/reverse-costing-reports/intel-realsense-d435-3d-active-ir-stereo-depth-camera/
This presentation explains the Transform coding in easiest method possible. The graphics and diagrammatic representations are worth looking for. Simple language is another pro.
Zipline is Airbnb’s data management platform specifically designed for ML use cases. Previously, ML practitioners at Airbnb spent roughly 60% of their time on collecting and writing transformations for machine learning tasks. Zipline reduces this task from months to days – by making the process declarative. It allows data scientists to easily define features in a simple configuration language. The framework then provides access to point-in-time correct features – for both – offline model training and online inference. In this talk we will describe the architecture of our system and the algorithm that makes the problem of efficient point-in-time correct feature generation, tractable.
The attendee will learn
Importance of point-in-time correct features for achieving better ML model performance
Importance of using change data capture for generating feature views
An algorithm – to efficiently generate features over change data. We use interval trees to efficiently compress time series features. The algorithm allows generating feature aggregates over this compressed representation.
A lambda architecture – that enables using the above algorithm – for online feature generation.
A framework, based on category theory, to understand how feature aggregations be distributed, and independently composed.
While the talk if fairly technical – we will introduce all the concepts from first principles with examples. Basic understanding of data-parallel distributed computation and machine learning might help, but are not required.
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Ijripublishers Ijri
Global interconnect planning becomes a challenge as semiconductor technology continuously scales. Because of the increasing wire resistance and higher capacitive coupling in smaller features, the delay of global interconnects becomes large compared with the delay of a logic gate, introducing a huge performance gap that needs to be resolved A novel equalized global link architecture and driver– receiver co design flow are proposed for high-speed and low-energy on-chip communication by utilizing a continuous-time linear equalizer (CTLE). The proposed global link is analyzed using a linear system method, and the formula of CTLE eye opening is derived to provide high-level design guidelines and insights.
Compared with the separate driver–receiver design flow, over 50% energy reduction is observed.
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Ijripublishers Ijri
Global interconnect planning becomes a challenge as semiconductor technology continuously scales. Because of the increasing wire resistance and higher capacitive coupling in smaller features, the delay of global interconnects becomes large compared with the delay of a logic gate, introducing a huge performance gap that needs to be resolved A novel equalized global link architecture and driver– receiver co design flow are proposed for high-speed and low-energy on-chip communication by utilizing a continuous-time linear equalizer (CTLE). The proposed global link is analyzed using a linear system method, and the formula of CTLE eye opening is derived to provide high-level design guidelines and insights.
Compared with the separate driver–receiver design flow, over 50% energy reduction is observed.
A fast-paced introduction to Deep Learning that starts with a simple yet complete neural network (no frameworks), followed by an overview of activation functions, cost functions, backpropagation, and then a quick dive into CNNs. Next we'll create a neural network using Keras, followed by an introduction to TensorFlow and TensorBoard. For best results, familiarity with basic vectors and matrices, inner (aka "dot") products of vectors, and rudimentary Python is definitely helpful.
Computer Vision: Correlation, Convolution, and GradientAhmed Gad
Three important operations in computer vision are explained starting with each one got explained and implemented in Python.
Generally, all of these three operations have many similarities in as they follow the same general steps but there are some subtle changes. The main change is using different masks.
A presentation on Human Activity Recognition catered to the audience from an HCI or CS background. (Based on research by Bulling, A. et al. 2014. A tutorial on human activity recognition using body-worn inertial sensors. CSUR. 46, 3 (2014), 33.)
Single Image Super-Resolution from Transformed Self-Exemplars (CVPR 2015)Jia-Bin Huang
Self-similarity based super-resolution (SR) algorithms are able to produce visually pleasing results without extensive training on external databases. Such algorithms exploit the statistical prior that patches in a natural image tend to recur within and across scales of the same image. However, the internal dictionary obtained from the given image may not always be sufficiently expressive to cover the textural appearance variations in the scene. In this paper, we extend self-similarity based SR to overcome this drawback. We expand the internal patch search space by allowing geometric variations. We do so by explicitly localizing planes in the scene and using the detected perspective geometry to guide the patch search process. We also incorporate additional affine transformations to accommodate local shape variations. We propose a compositional model to simultaneously handle both types of transformations. We extensively evaluate the performance in both urban and natural scenes. Even without using any external training databases, we achieve significantly superior results on urban scenes, while maintaining comparable performance on natural scenes as other state-of-the-art SR algorithms.
http://bit.ly/selfexemplarsr
발표자: 이인웅 (연세대 박사과정)
발표일: 2017.12.
개요:
영상에서 사람의 행동을 인식하는 방법은 크게 영상에서 직접적으로 행동 라벨을 추출하는 것과 자세 정보를 기반으로 행동 라벨을 추출하는 것으로 나뉠 수 있습니다.
본 발표는 행동 인식에 대한 전반적인 개요를 설명하고 그 중에서도 사람의 자세 정보를 기반으로 하는 행동 인식 기술에 초점을 두고 최근 ICCV 2017 학회에서 발표된 Temporal Sliding LSTM 네트워크를 활용한 행동 인식 기술을 중점적으로 설명합니다. 구체적으로, 스켈레톤 기반 행동 인식 이슈, 제안하는 방법과 실험 결과들이 소개되고 앞으로 나아갈 만한 새로운 연구 이슈들도 추가적으로 설명합니다.
Ukrainian Catholic University
Faculty of Applied Sciences
Data Science Master Program
January 22nd
Abstract. The thesis introduces the reader to the concepts of edge computing in terms of person re-identification and tracking problem. It describes the challenges, limitations, and current state-of-the-art solutions. The author proposed a pipeline for the task, launched several experiments on validating different parts of the system, and provided a theoretical explanation of the person re-identification process in the overlapping multi-camera environment.
Intel RealSense D435 3D Active IR Stereo Depth Camera 2018 teardown reverse c...system_plus
The 3D camera is using infrared active stereo depth and red/green/blue sensors, and a VCSEL projector.
More information on that report at http://www.systemplus.fr/reverse-costing-reports/intel-realsense-d435-3d-active-ir-stereo-depth-camera/
This presentation explains the Transform coding in easiest method possible. The graphics and diagrammatic representations are worth looking for. Simple language is another pro.
Zipline is Airbnb’s data management platform specifically designed for ML use cases. Previously, ML practitioners at Airbnb spent roughly 60% of their time on collecting and writing transformations for machine learning tasks. Zipline reduces this task from months to days – by making the process declarative. It allows data scientists to easily define features in a simple configuration language. The framework then provides access to point-in-time correct features – for both – offline model training and online inference. In this talk we will describe the architecture of our system and the algorithm that makes the problem of efficient point-in-time correct feature generation, tractable.
The attendee will learn
Importance of point-in-time correct features for achieving better ML model performance
Importance of using change data capture for generating feature views
An algorithm – to efficiently generate features over change data. We use interval trees to efficiently compress time series features. The algorithm allows generating feature aggregates over this compressed representation.
A lambda architecture – that enables using the above algorithm – for online feature generation.
A framework, based on category theory, to understand how feature aggregations be distributed, and independently composed.
While the talk if fairly technical – we will introduce all the concepts from first principles with examples. Basic understanding of data-parallel distributed computation and machine learning might help, but are not required.
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Ijripublishers Ijri
Global interconnect planning becomes a challenge as semiconductor technology continuously scales. Because of the increasing wire resistance and higher capacitive coupling in smaller features, the delay of global interconnects becomes large compared with the delay of a logic gate, introducing a huge performance gap that needs to be resolved A novel equalized global link architecture and driver– receiver co design flow are proposed for high-speed and low-energy on-chip communication by utilizing a continuous-time linear equalizer (CTLE). The proposed global link is analyzed using a linear system method, and the formula of CTLE eye opening is derived to provide high-level design guidelines and insights.
Compared with the separate driver–receiver design flow, over 50% energy reduction is observed.
Jiri ece-01-03 adaptive temporal averaging and frame prediction based surveil...Ijripublishers Ijri
Global interconnect planning becomes a challenge as semiconductor technology continuously scales. Because of the increasing wire resistance and higher capacitive coupling in smaller features, the delay of global interconnects becomes large compared with the delay of a logic gate, introducing a huge performance gap that needs to be resolved A novel equalized global link architecture and driver– receiver co design flow are proposed for high-speed and low-energy on-chip communication by utilizing a continuous-time linear equalizer (CTLE). The proposed global link is analyzed using a linear system method, and the formula of CTLE eye opening is derived to provide high-level design guidelines and insights.
Compared with the separate driver–receiver design flow, over 50% energy reduction is observed.
IJERA (International journal of Engineering Research and Applications) is International online, ... peer reviewed journal. For more detail or submit your article, please visit www.ijera.com
MPEC2: Multilayer and Pipeline Video Encoding on the Computing ContinuumAlpen-Adria-Universität
Video streaming is the dominating traffic in today’s data-sharing world. Media service providers stream video content for their viewers, while worldwide users create and distribute videos using mobile or video system applications that significantly increase the traffic share. We propose a multilayer and pipeline encoding on the computing continuum (MPEC2) method that addresses the key technical challenge of high-price and computational complexity of video encoding. MPEC2 splits the video encoding into several tasks scheduled on appropriately selected Cloud and Fog computing instance types that satisfy the media service provider and user priorities in terms of time and cost.
In the first phase, MPEC2 uses a multilayer resource partitioning method to explore the instance types for encoding a video segment. In the second phase, it distributes the independent segment encoding tasks in a pipeline model on the underlying instances.
We evaluate MPEC2 on a federated computing continuum encompassing Amazon Web Services (AWS) EC2 Cloud and Exoscale Fog instances distributed on seven geographical locations. Experimental results show that MPEC2 achieves 24% faster completion time and 60% lower cost for video encoding compared to resource allocation related methods. When compared with baseline methods, MPEC2 yields 40%-50% lower completion time and 5-60% reduced total cost.
Video traffic on the Internet is constantly growing; networked multimedia applications consume a predominant share of the available Internet bandwidth. A major technical breakthrough and enabler in multimedia systems research and of industrial networked multimedia services certainly was the HTTP Adaptive Streaming (HAS) technique. This resulted in the standardization of MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH) which, together with HTTP Live Streaming (HLS), is widely used for multimedia delivery in today’s networks. Existing challenges in multimedia systems research deal with the trade-off between (i) the ever-increasing content complexity, (ii) various requirements with respect to time (most importantly, latency), and (iii) quality of experience (QoE). Optimizing towards one aspect usually negatively impacts at least one of the other two aspects if not both. This situation sets the stage for our research work in the ATHENA Christian Doppler (CD) Laboratory (Adaptive Streaming over HTTP and Emerging Networked Multimedia Services; https://athena.itec.aau.at/), jointly funded by public sources and industry. In this talk, we will present selected novel approaches and research results of the first year of the ATHENA CD Lab’s operation. We will highlight HAS-related research on (i) multimedia content provisioning (machine learning for video encoding); (ii) multimedia content delivery (support of edge processing and virtualized network functions for video networking); (iii) multimedia content consumption and end-to-end aspects (player-triggered segment retransmissions to improve video playout quality); and (iv) novel QoE investigations (adaptive point cloud streaming). We will also put the work into the context of international multimedia systems research.
International Journal of Engineering Research and Applications (IJERA) is an open access online peer reviewed international journal that publishes research and review articles in the fields of Computer Science, Neural Networks, Electrical Engineering, Software Engineering, Information Technology, Mechanical Engineering, Chemical Engineering, Plastic Engineering, Food Technology, Textile Engineering, Nano Technology & science, Power Electronics, Electronics & Communication Engineering, Computational mathematics, Image processing, Civil Engineering, Structural Engineering, Environmental Engineering, VLSI Testing & Low Power VLSI Design etc.
Analyzing Video Streaming Quality by Using Various Error Correction Methods o...IJERA Editor
Transmission video over ad hoc networks has become one of the most important and interesting subjects of study for researchers and programmers because of the strong relationship between video applications and frequent users of various mobile devices, such as laptops, PDAs, and mobile phones in all aspects of life. However, many challenges, such as packet loss, congestion (i.e., impairments at the network layer), multipath fading (i.e., impairments at the physical layer) [1], and link failure, exist in transferring video over ad hoc networks; these challenges negatively affect the quality of the perceived video [2].This study has investigated video transfer over ad hoc networks. The main challenges of transferring video over ad hoc networks as well as types of errors that may occur during video transmission, various types of video mechanisms, error correction methods, and different Quality of Service (QoS) parameters that affect the quality of the received video are also investigated.
Design and Analysis of Quantization Based Low Bit Rate Encoding Systemijtsrd
The objective of this paper is to develop a low bit rate encoding for VQ problems such as real time image coding.. The decision tree is generated by an offline process.. A new systolic architecture to realize the encoder of full search vector quantization VQ for high speed applications is presented here. Over past decades digital video compression technologies have become an integral part. Therefore the purpose is to improve image quality in Remote cardiac pulse measurement using Adaptive filter. It describes the approach to be used for feature extraction from many images.. This paper presents a real time application of compression of the image processing technique which can be efficiently used for the interfacing with any hardware. Therefore we have used Raspberry Pi in compression of image. We have developed an algorithm that is based on the endoscopic images that consist of the differential pulse code modulation. The compressors consist of a low cost YEF colour space converters and variable length predictive algorithm for lossless compression. Mr. Nilesh Bodne | Dr. Sunil Kumar "Design and Analysis of Quantization Based Low Bit Rate Encoding System" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-6 , October 2019, URL: https://www.ijtsrd.com/papers/ijtsrd29289.pdf Paper URL: https://www.ijtsrd.com/engineering/electronics-and-communication-engineering/29289/design-and-analysis-of-quantization-based-low-bit-rate-encoding-system/mr-nilesh-bodne
VE-Match: Video Encoding Matching-based Model for Cloud and Edge Computing In...Alpen-Adria-Universität
The considerable surge in energy consumption within data centers can be attributed to the exponential rise in demand for complex computing workflows and storage resources. Video streaming applications are both compute and storage-intensive and account for the majority of today’s internet services. In this work, we designed a video encoding application consisting of codec, bitrate, and resolution set for encoding a video segment. Then, we propose VE-Match, a matching-based method to schedule video encoding applications on both Cloud and Edge resources to optimize costs and energy consumption. Evaluation results on a real computing testbed federated between Amazon Web Services (AWS) EC2 Cloud instances and the Alpen-Adria University (AAU) Edge server reveal that VE-Match achieves lower costs by 17%-78% in the cost-optimized scenarios compared to the energy-optimized and tradeoff between cost and energy. Moreover, VE-Match improves the video encoding energy consumption by 38%-45% and gCO2 emission by up to 80 % in the energy-optimized scenarios compared to the cost-optimized and tradeoff between cost and energy.
Deep learning-based switchable network for in-loop filtering in high efficie...IJECEIAES
The video codecs are focusing on a smart transition in this era. A future area of research that has not yet been fully investigated is the effect of deep learning on video compression. The paper’s goal is to reduce the ringing and artifacts that loop filtering causes when high-efficiency video compression is used. Even though there is a lot of research being done to lessen this effect, there are still many improvements that can be made. In This paper we have focused on an intelligent solution for improvising in-loop filtering in high efficiency video coding (HEVC) using a deep convolutional neural network (CNN). The paper proposes the design and implementation of deep CNN-based loop filtering using a series of 15 CNN networks followed by a combine and squeeze network that improves feature extraction. The resultant output is free from double enhancement and the peak signal-to-noise ratio is improved by 0.5 dB compared to existing techniques. The experiments then demonstrate that improving the coding efficiency by pipelining this network to the current network and using it for higher quantization parameters (QP) is more effective than using it separately. Coding efficiency is improved by an average of 8.3% with the switching based deep CNN in-loop filtering.
AI firsts: Leading from research to proof-of-conceptQualcomm Research
AI has made tremendous progress over the past decade, with many advancements coming from fundamental research from many decades ago. Accelerating the pipeline from research to commercialization has been daunting since scaling technologies in the real world faces many challenges beyond the theoretical work done in the lab. Qualcomm AI Research has taken on the task of not only generating novel AI research but also being first to demonstrate proof-of-concepts on commercial devices, enabling technology to scale in the real world. This presentation covers:
The challenges of deploying cutting-edge research on real-world mobile devices
How Qualcomm AI Research is solving system and feasibility challenges with full-stack optimizations to quickly move from research to commercialization
Examples where Qualcomm AI Research has had industrial or academic firsts
Nutraceutical market, scope and growth: Herbal drug technologyLokesh Patil
As consumer awareness of health and wellness rises, the nutraceutical market—which includes goods like functional meals, drinks, and dietary supplements that provide health advantages beyond basic nutrition—is growing significantly. As healthcare expenses rise, the population ages, and people want natural and preventative health solutions more and more, this industry is increasing quickly. Further driving market expansion are product formulation innovations and the use of cutting-edge technology for customized nutrition. With its worldwide reach, the nutraceutical industry is expected to keep growing and provide significant chances for research and investment in a number of categories, including vitamins, minerals, probiotics, and herbal supplements.
Deep Behavioral Phenotyping in Systems Neuroscience for Functional Atlasing a...Ana Luísa Pinho
Functional Magnetic Resonance Imaging (fMRI) provides means to characterize brain activations in response to behavior. However, cognitive neuroscience has been limited to group-level effects referring to the performance of specific tasks. To obtain the functional profile of elementary cognitive mechanisms, the combination of brain responses to many tasks is required. Yet, to date, both structural atlases and parcellation-based activations do not fully account for cognitive function and still present several limitations. Further, they do not adapt overall to individual characteristics. In this talk, I will give an account of deep-behavioral phenotyping strategies, namely data-driven methods in large task-fMRI datasets, to optimize functional brain-data collection and improve inference of effects-of-interest related to mental processes. Key to this approach is the employment of fast multi-functional paradigms rich on features that can be well parametrized and, consequently, facilitate the creation of psycho-physiological constructs to be modelled with imaging data. Particular emphasis will be given to music stimuli when studying high-order cognitive mechanisms, due to their ecological nature and quality to enable complex behavior compounded by discrete entities. I will also discuss how deep-behavioral phenotyping and individualized models applied to neuroimaging data can better account for the subject-specific organization of domain-general cognitive systems in the human brain. Finally, the accumulation of functional brain signatures brings the possibility to clarify relationships among tasks and create a univocal link between brain systems and mental functions through: (1) the development of ontologies proposing an organization of cognitive processes; and (2) brain-network taxonomies describing functional specialization. To this end, tools to improve commensurability in cognitive science are necessary, such as public repositories, ontology-based platforms and automated meta-analysis tools. I will thus discuss some brain-atlasing resources currently under development, and their applicability in cognitive as well as clinical neuroscience.
Earliest Galaxies in the JADES Origins Field: Luminosity Function and Cosmic ...Sérgio Sacani
We characterize the earliest galaxy population in the JADES Origins Field (JOF), the deepest
imaging field observed with JWST. We make use of the ancillary Hubble optical images (5 filters
spanning 0.4−0.9µm) and novel JWST images with 14 filters spanning 0.8−5µm, including 7 mediumband filters, and reaching total exposure times of up to 46 hours per filter. We combine all our data
at > 2.3µm to construct an ultradeep image, reaching as deep as ≈ 31.4 AB mag in the stack and
30.3-31.0 AB mag (5σ, r = 0.1” circular aperture) in individual filters. We measure photometric
redshifts and use robust selection criteria to identify a sample of eight galaxy candidates at redshifts
z = 11.5 − 15. These objects show compact half-light radii of R1/2 ∼ 50 − 200pc, stellar masses of
M⋆ ∼ 107−108M⊙, and star-formation rates of SFR ∼ 0.1−1 M⊙ yr−1
. Our search finds no candidates
at 15 < z < 20, placing upper limits at these redshifts. We develop a forward modeling approach to
infer the properties of the evolving luminosity function without binning in redshift or luminosity that
marginalizes over the photometric redshift uncertainty of our candidate galaxies and incorporates the
impact of non-detections. We find a z = 12 luminosity function in good agreement with prior results,
and that the luminosity function normalization and UV luminosity density decline by a factor of ∼ 2.5
from z = 12 to z = 14. We discuss the possible implications of our results in the context of theoretical
models for evolution of the dark matter halo mass function.
Remote Sensing and Computational, Evolutionary, Supercomputing, and Intellige...University of Maribor
Slides from talk:
Aleš Zamuda: Remote Sensing and Computational, Evolutionary, Supercomputing, and Intelligent Systems.
11th International Conference on Electrical, Electronics and Computer Engineering (IcETRAN), Niš, 3-6 June 2024
Inter-Society Networking Panel GRSS/MTT-S/CIS Panel Session: Promoting Connection and Cooperation
https://www.etran.rs/2024/en/home-english/
Phenomics assisted breeding in crop improvementIshaGoswami9
As the population is increasing and will reach about 9 billion upto 2050. Also due to climate change, it is difficult to meet the food requirement of such a large population. Facing the challenges presented by resource shortages, climate
change, and increasing global population, crop yield and quality need to be improved in a sustainable way over the coming decades. Genetic improvement by breeding is the best way to increase crop productivity. With the rapid progression of functional
genomics, an increasing number of crop genomes have been sequenced and dozens of genes influencing key agronomic traits have been identified. However, current genome sequence information has not been adequately exploited for understanding
the complex characteristics of multiple gene, owing to a lack of crop phenotypic data. Efficient, automatic, and accurate technologies and platforms that can capture phenotypic data that can
be linked to genomics information for crop improvement at all growth stages have become as important as genotyping. Thus,
high-throughput phenotyping has become the major bottleneck restricting crop breeding. Plant phenomics has been defined as the high-throughput, accurate acquisition and analysis of multi-dimensional phenotypes
during crop growing stages at the organism level, including the cell, tissue, organ, individual plant, plot, and field levels. With the rapid development of novel sensors, imaging technology,
and analysis methods, numerous infrastructure platforms have been developed for phenotyping.
ESR spectroscopy in liquid food and beverages.pptxPRIYANKA PATEL
With increasing population, people need to rely on packaged food stuffs. Packaging of food materials requires the preservation of food. There are various methods for the treatment of food to preserve them and irradiation treatment of food is one of them. It is the most common and the most harmless method for the food preservation as it does not alter the necessary micronutrients of food materials. Although irradiated food doesn’t cause any harm to the human health but still the quality assessment of food is required to provide consumers with necessary information about the food. ESR spectroscopy is the most sophisticated way to investigate the quality of the food and the free radicals induced during the processing of the food. ESR spin trapping technique is useful for the detection of highly unstable radicals in the food. The antioxidant capability of liquid food and beverages in mainly performed by spin trapping technique.
ANAMOLOUS SECONDARY GROWTH IN DICOT ROOTS.pptxRASHMI M G
Abnormal or anomalous secondary growth in plants. It defines secondary growth as an increase in plant girth due to vascular cambium or cork cambium. Anomalous secondary growth does not follow the normal pattern of a single vascular cambium producing xylem internally and phloem externally.
Richard's aventures in two entangled wonderlandsRichard Gill
Since the loophole-free Bell experiments of 2020 and the Nobel prizes in physics of 2022, critics of Bell's work have retreated to the fortress of super-determinism. Now, super-determinism is a derogatory word - it just means "determinism". Palmer, Hance and Hossenfelder argue that quantum mechanics and determinism are not incompatible, using a sophisticated mathematical construction based on a subtle thinning of allowed states and measurements in quantum mechanics, such that what is left appears to make Bell's argument fail, without altering the empirical predictions of quantum mechanics. I think however that it is a smoke screen, and the slogan "lost in math" comes to my mind. I will discuss some other recent disproofs of Bell's theorem using the language of causality based on causal graphs. Causal thinking is also central to law and justice. I will mention surprising connections to my work on serial killer nurse cases, in particular the Dutch case of Lucia de Berk and the current UK case of Lucy Letby.
This presentation explores a brief idea about the structural and functional attributes of nucleotides, the structure and function of genetic materials along with the impact of UV rays and pH upon them.
1. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Study and Implementation of an Object-based Video Encoder
for Embedded Wireless Video Surveillance Systems
Thesis defense
Aliouat Ahcen
Supervisors: Dr. Nasreddine Kouadria & Dr. Saliha Harize
LASA Laboratory, Electronics Department, Faculty of Technology, Badji Mokhtar - Annaba University
June 12, 2023
Ahcen Badji Mokhtar - Annaba University 1 / 91
2. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 2 / 91
3. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 3 / 91
4. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Working framework
1 In our problem, we consider a surveillance system that uses a wireless multimedia
sensor network (WMSN) as a backbone for capturing and delivering multimedia
data.
2 We consider also Wireless connections which have challenges in terms of bandwidth
requirement and energy consumption.
3 We are addressing this problem by developing low-cost pre-encoders to reduce the
overall cost of the video encoder in terms of bitrate and energy consumption.
1
1Conservation X Labs product (Edge Cloud AI solution)
Ahcen Badji Mokhtar - Annaba University 4 / 91
5. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Framework
This thesis has been conducted as part of the Franco-Algerian Cooperation
project PHC Tassili.
The PHC Tassili project aims to propose solutions for migratory waterbird
monitoring using WMSN and Artificial Intelligence (AI).
This project proposes a combination of image, video, and audio solutions.
In this scope, the thesis is contributing in the project by detecting and
compressing birds’ ROIs prior to transmission.
PHC Tassili project
Ahcen Badji Mokhtar - Annaba University 5 / 91
6. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Visual Sensor Node
Wirless Multimedia Sensor Network
Transmission
Network
Display System
wireless Link wire Link
Nodes continually capture images / Equipped with batteries (limited energy
source)/ Wireless communication.
Advantages: Surveillance using WMSN
Their ability to cover critical and far zones (military, wild, lakes..) without intervention /
Cover larger zones / Real-time communication of the data / Cooperation of network
nodes
Challenges: WMSN
High data size / Limited energy / Limited bandwidth / High network congestion
Lets consider one sensor node. . .
Ahcen Badji Mokhtar - Annaba University 6 / 91
7. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Wireless Visual Sensor Node
C
a
p
t
u
r
e
d
F
r
a
m
e
Standard
Video / Imgae
Encoder
Buffering and
Radio Transmission
Compressed
Data
The whole frame
High Energy Consumption
High data rate
Fast battery drop
Equal priority to important and non important regions in the frame
Bitstream
Coding efficiency influences directly: Energy/bitrate/memory usage/image quality
The standard approach: processing the whole frame equally, ∀ blocks ,
without priority.
Alternatives: Adding a pre-processing step before performing compression,
called: Region of Interest detection step
Ahcen Badji Mokhtar - Annaba University 7 / 91
8. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Image/video coding in sensor node
Considering a ROI detection as pre-encoder for video compression
The pre-processing step is an aid to the encoder to achieve the desired tradeoff.
fdfd
gfgg
fgfgf
To overcome the challenges of complexity/quality/bitrate trade-off, the
encoder must :
Be source side-friendly (the sensor node as a source).
Ensure very low bitrate output.
Achieve an acceptable frame rate.
dfdf
fdfd
gfgg
fgfgf
Applying a ROI detection means applying moving object detection in the video
sequence . . . So, what are moving object detection approaches?
Ahcen Badji Mokhtar - Annaba University 8 / 91
9. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Moving object detection in video sequence
background model
F(n - 1)
Background Subtraction
Frame Difference
Edge Detection
F(n)
F(n)
F(n)
F(n - 1)
Other techniques
Most of the other techniques are a combination or variant of those methods
Ahcen Badji Mokhtar - Annaba University 9 / 91
10. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Moving Object as Region of Interest (ROI)
What is a region of Interest?
In a video frame, the different regions are not of the same interest
The human eye is interested in the object in the frame, either the moving or the still
object.
Example: ROI can include
A pedestrian walking / a car in the street / a flying bird / any object that creates
movement between frames
Moving Objects as Region of Interest
How to process the ROI?
Block based processing of the ROI is better for compression, which allow achieving high
detection accuracy.
Ahcen Badji Mokhtar - Annaba University 10 / 91
11. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Impact of ROI detection on compression
To encode frame based on ROIs
We are trying to avoid energy/bitrate wasting in encoding unnecessary data.
Unnecessary data are those blocks with no or negligible changes.
Benefit of coding the frame based on ROI
Important gain in data rate and energy / Achieving real-time conditions with High
ROI quality.
What are the conditions?
High accuracy in detecting all the moving regions to avoid artifacts.
R
O
I
n
o
n
-
R
O
I
Wireless Visual Sensor Node
C
a
p
t
u
r
e
d
F
r
a
m
e
Pre-encoder
(ROI Detector)
ROI-based
Video Encoder
ROI
Recommendation ROI: Region-Of-Interest
Buffering
and Transmission
Compressed
Data
The research question then arises...
Ahcen Badji Mokhtar - Annaba University 11 / 91
12. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Research question of the thesis
Video/Image
Coding
Wireless
Multimedia
Sensor Network
ROI detection for video
coding in WMSN
(our approachs)
Object
Detection
(Region-of-Interest)
E
n
e
r
g
y
Network lifetime/rate Accuracy
C
o
m
p
l
e
x
i
t
y
Q
o
S
/
Q
o
E
B
i
t
r
a
t
e
Research question
How can we detect ROI in a captured video to ensure high-quality encoding and
transmission over a WMSN while minimizing bitrate and energy consumption?
The context and objective are then clear...
Ahcen Badji Mokhtar - Annaba University 12 / 91
13. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Expected results from this thesis
R
O
I
n
o
n
-
R
O
I
Wireless Visual Sensor Node (Transmitter)
C
a
p
t
u
r
e
d
F
r
a
m
e
Pre-encoder
ROI Detector
ROI-based
Video Encoder
ROI
Recommendation ROI: Region-Of-Interest
Buffering and
Radio Transmission
Compressed
Data
Video Analysis
(Decision?)
Video Decoder
based on ROI
ROI: Region-Of-Interest
Receiver
Compressed
Data
Channel Conditions
Bitstream
Decide
Recognize
Destination (Receiver)
Classify
Monitor
Recommand
Recovered
Data
Overall scheme of the thesis contribution conditions
Ahcen Badji Mokhtar - Annaba University 13 / 91
14. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Background Background Background
Organization of the contributions
Thesis
Contributions plan
Binary
Classification
Multi-class
Classification
Contribution 1
Contribution 2
Contribution 3
Contribution 4
PART 1
PART 2
Ahcen Badji Mokhtar - Annaba University 14 / 91
15. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 15 / 91
16. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Related work: ROI based video coding
1 Kouadria et al.: ‘Region-of-interest based image compression using the discrete
tchebichef transform in wireless visual sensor networks’ - 2019.
Detect and transmit only the ROI using SAD.
Gain:
Very low bitrate (about 2kB needed for an image of size 320x360).
Very low complexity adapted for WMSN.
Limits:
Limited accuracy of the ROI detection algorithm.
Validated on small dataset/Limited number of evaluation metrics.
Ahcen Badji Mokhtar - Annaba University 16 / 91
17. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Related works: ROI based video coding
2 Rehman et al.: “A novel energy efficient object detection and image transmission
approach for wireless multimedia sensor networks” – 2016.
Separate the frame into 4 blocks and transmit only the active blocks.
Gain:
Moderate bitrate for transmission with simple detection.
Limits:
High detection and compression complexity.
Validated on small dataset.
Limited number of evaluation metrics.
Can be optimized to have lower bitrate.
Ahcen Badji Mokhtar - Annaba University 17 / 91
18. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 18 / 91
19. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Used Dataset over the contributions
Surveillance datasets used in our work experiments.
Work State / Source Number of sequence
Contribution 1 [1] Multiple sequence / Multiple Dataset 9 video sequences
Contribution 2 [2] Complete dataset : CDnet 2014 51 video (15000 frame)
Contribution 3 [3] Multiple sequences / Multiple Dataset 3 video sequences
Contribution 4 [4] Multiple sequence / Multiple Dataset 9 video sequences
Condition of the captured scences of the datasets
Indoor/Outdoor surveillance sequences.
Human, highway, pedestrians, battlefield . . . objects are contained in the sequences.
Color, gray-scale, thermal images.
Weather conditions: rain, snowfall (noisy background), sunny . . .
QCIF, CIF, . . . HD resolutions.
Night and day time capturing.
Ahcen Badji Mokhtar - Annaba University 19 / 91
20. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Embedded environment conditions
for the sake of precise validation, an embedded environment conditions are applied.
We assume an STM32 ARM cortex M3 motherboard as an embedded system.
The energy consumption of basic arithmetic operations is considered
(addition/subtraction/division/multiplication).
STM32 ARM Cortex M3 characteristics (contributions [2] and [4])
Sensor Processor Cortex M3
Clock rate 72 MHz
Processor power 23 mW
Cycles count Add. (1), Sub.(1), Mult.(1 or 2), Div.(1 to 12).
Ahcen Badji Mokhtar - Annaba University 20 / 91
21. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Performance evaluation and used metrics
Metric
PSNR | SSIM | MS-SSIM | VIF | Balanced-Accuracy |
Recall | Precision | Sensitivity | Specificity | FPR
| FNR| PWC | TP | FP | TN | FN | F-measure
Value/Score
Reference Frame / Original Frame/ Ground Trouth
Resulted Frame
Evaluation metrics used.
Ahcen Badji Mokhtar - Annaba University 21 / 91
22. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 22 / 91
24. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Model/Method
We proposed an ROI detection method based on Edge Detection + Frame
Difference
The detection is enhanced by the ROF+FGS filter
The method resolve the error propagation problem by proposing error
correction approach.
Block diagram of the proposed ROI detection method
Ahcen Badji Mokhtar - Annaba University 24 / 91
25. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Model/Method
proposed algorithm for ROI
detection/compression
Using Sobel for edge detection
Using SAD on edge feature
2-D ROF and FGS to enhance
Error propagation avoidance by
whole frame compression after
each GOP
Compress and transmit
only the ROI blocks
Receive, decode,
and update the display
Sobel Edge Detector
Input Frame n
Input Frame n-1
Edge Difference map Calculator
Sum of 4x4 Elements (SAD)
2-D Rank Odrer Filter of 8x8 window
Fast Global Smoother
Score Threshold? binary mask(block) = 0
binary mask(block) = 1
No
Yes
Compress then Transmit the Block
Skip the Block
GOP acheaved?
No
Yes
Compress
and Transmit
the whole Frame
Input Frame n-1 Input Frame n
Ahcen Badji Mokhtar - Annaba University 25 / 91
26. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Results and discussion
ROI detection results
The ROI mask includes all the objects, the results show high accuracy for ROI detection.
Ahcen Badji Mokhtar - Annaba University 26 / 91
27. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Quality evaluation
PSNR, SSIM, MS-SSIM and VIF results for the used dataset
SSIM, MS-SSIM, PSNR and VIF for: atrium
Ahcen Badji Mokhtar - Annaba University 27 / 91
28. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Data reduction
Mean number of blocks to be transmitted for each strategy
Sequence
name
Sequence
size
ROI-based
(ours)
Classical
approach
Saving(%)
(wr. to classical)
Traffic2 640x360 4695 14400 67.4%
Atrium 640x360 589 14400 96%
Highway 320x240 1345 4800 72%
freeway 316x236 530 4661 88.6%
peds 232x152 719 2204 67.4%
rain 308x228 2132 4389 51.5%
traffic 378x282 1768 6662 73.5%
traffic3 160x120 428 1200 64.3%
Advantage
Energy saving: between 51% to 96%.
Ahcen Badji Mokhtar - Annaba University 28 / 91
29. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Limits in terms of visual Quality
Missing Information
due to wrong moving region detection
Blocks to be transmitted
Corect detection
(white pixels)
Error Probagation
due to continues wrong
object detection
Blocks to be Skipped
no information change
Between each GOP, a wrong detection of the ROI leads to a propagation of the visual
artifacts.
Ahcen Badji Mokhtar - Annaba University 29 / 91
30. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Limits in the context of ROI detection for video
coding in WMSN
Some limits
1 High energy consumption is expected due to the used edge detection method.
2 Limited size of the used dataset
3 Quantitative evaluation of the detection performances is not performed
4 The energy consumption in an embedded environment is not evaluated.
5 A comparison to the state of the art is not shown.
How to solve this?
The next contribution shows a very low energy consumption method evaluated on
a large corpus dataset while the energy consumption is modeled and the results
are compared to the state of the art.
Ahcen Badji Mokhtar - Annaba University 30 / 91
31. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion Recap.
Recap.
The proposed approach is efficient when applied in surveillance camera.
Edge feature detection and error correction using fixed GOP intervals.
Sobel edge detection is effective in identifying frame changes, and SAD
accurately locates the ROI.
Achieves significant bandwidth savings, energy reduction (51.5%-96%), and
high-quality frame reconstruction.
Sobel edge detector consumes excessive energy in sensor node.
The study lucks: Accuracy and energy consumption analysis.
The upcoming contribution will address these limitations.
Ahcen Badji Mokhtar - Annaba University 31 / 91
32. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 32 / 91
33. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Title: ”An Efficient Low Complexity Region-of-Interest Detection for Video Coding
in Wireless Visual Surveillance” IEEE Access, IF=3.41.
Ahcen Badji Mokhtar - Annaba University 33 / 91
34. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Main contribution
Block-based movIng Region Detection (BIRD)
Method: Low complexity ROI detection method for video coding in constrained WVS.
Accuracy: Improved detection accuracy through a combination of fast Gaussian
smoother and rank-order filter.
Performance: Algorithm assessed using several metrics to evaluate detection
performance and confirm superiority over SOTA techniques in constrained WVS.
Benefits: Bitrate and energy savings achieved using algorithm as a pre-encoder of a
baseline JPEG compression chain.
Viability: Algorithm’s viability for implementation in WVS demonstrated based on
energy/memory consumption modeling using ARM Cortex M3 characteristics.
R
O
I
n
o
n
-
R
O
I
Wireless Visual Sensor Node
C
a
p
t
u
r
e
d
F
r
a
m
e
Pre-encoder
(ROI Detector)
ROI-based
Video Encoder
ROI
Recommendation ROI: Region-Of-Interest
Buffering
and Transmission
Compressed
Data
Accurate pre-encoder to encode only moving frames
Ahcen Badji Mokhtar - Annaba University 34 / 91
35. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Proposed Method
Block diagram of the proposed algorithm (BIRD)
SFD : ϕn(x, y) =
1
w2
w−1
X
u=0
w−1
X
v=0
Fn(wx + u, wy + v), (1)
Ahcen Badji Mokhtar - Annaba University 35 / 91
36. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Proposed method
Difference between maps:
∆(w, y) = |ϕn(x, y) − ϕm(x, y)| (2)
Impact of FGS and ROF
Ahcen Badji Mokhtar - Annaba University 36 / 91
37. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Performances of BIRD over the CDnet 2014
Visual results (binary mask): Recall (TPR) is optimized to cover all objects (ROI)
A. Aliouat et al.: Region-of-Interest Detection for Wireless Visual Surveillance
TABLE 3: Samples of ROI extraction mask results
Sequence Original ground-truth mask ROI
Highway #1475
SnowFall #2784
Pedestrians #476
Blizzard #1406
WinterDriveway
#1860
tunnelExit #2329
Sofa #1185
PTZ #1240 Ahcen Badji Mokhtar - Annaba University 37 / 91
39. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Quantitative results: Compared to SOTA
Technique Recall Specificity FPR FNR PWC F-Measure Precision
KNN [1] 0.6650 0.9802 0.0198 0.3350 3.3200 0.5937 0.6788
GMM1 [2] 0.6846 0.9750 0.0250 0.3154 3.7667 0.5707 0.6025
KDE [3] 0.7375 0.9519 0.0481 0.2625 5.6262 0.5688 0.5811
MahaD [4] 0.1644 0.9931 0.0069 0.8356 3.4750 0.2267 0.7403
GMM2 [5] 0.6604 0.9725 0.0275 0.3396 3.9953 0.5566 0.5973
EucD [4] 0.6803 0.9449 0.0551 0.3197 6.5423 0.5161 0.5480
BIRD 0.8084 0.8357 0.1642 0.1915 16.7115 0.1893 0.2678
Comparison of BIRD with classical techniques over CDnet 2014 dataset
Ahcen Badji Mokhtar - Annaba University 39 / 91
40. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Quantitative results:: Comparison with SOTA
Category-wise comparison of BIRD with SOTA on CDnet 2014 dataset
Category
Recall Specificity Balanced Acc.
BIRD Savas[6] Cwizar[7] BIRD Savas[6] Cwizar[7] BIRD Savas [6] Cwizar[7]
Dynamic. 0.7593 0.6436 0.8144 0.9512 0.9962 0.9985 0.8553 0.8199 0.9064
PTZ 0.9662 0.7685 0.3833 0.6443 0.9977 0.9968 0.8053 0.8831 0.6901
BadWeat. 0.9208 0.5647 0.6697 0.8948 0.9985 0.9993 0.9078 0.7816 0.8345
Baseline 0.7619 0.6214 0.8972 0.9437 0.8213 0.9980 0.8528 0.7213 0.9476
CameraJ. 0.8504 0.4567 0.7436 0.6446 0.9788 0.9931 0.7475 0.7177 0.8683
Intermi. 0.4186 0.5547 0.8324 0.8603 0.9979 0.9911 0.6394 0.7763 0.9118
LowFram. 0.8161 0.5490 0.6659 0.7905 0.7464 0.9949 0.8033 0.6477 0.8304
nightVi. 0.9455 0.4593 0.4511 0.8374 0.9583 0.9874 0.8915 0.7088 0.7193
Shadow 0.8775 0.8365 0.8786 0.8500 0.9828 0.9910 0.8638 0.9097 0.9348
Thermal 0.7548 0.4650 0.7268 0.8894 0.9647 0.9949 0.8221 0.7148 0.8609
Turbule. 0.8216 0.7421 0.7122 0.8870 0.9883 0.9997 0.8543 0.8652 0.8559
Overall 0.8084 0.6056 0.6608 0.8357 0.9483 0.9948 0.8220 0.7770 0.8509
*bold values are the best category-wise, red values are the best overall, blue values are the second best
Ahcen Badji Mokhtar - Annaba University 40 / 91
41. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Energy consumption
The total energy consumption in the node is equal to:
Etotal = EDetection + Ecompress, (3)
While Ecompress is estimated from [8]2
, EDetection is equal to:
EDetection = ESF D + EF GS + EROF + ET hreshold (4)
2Energy-efficient image compression for resource-constrained platforms, Lee Dong-U et al.,
IEEE Transactions on Image Processing,2009.
Ahcen Badji Mokhtar - Annaba University 41 / 91
42. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Energy consumption: proportion of blocks
considered for transmission, and the gain
Statistics of the energy gain variable threshold values.
Threshold Highway Pedestrians Snowfall
∆ energy ∆ energy ∆ energy
- mean (ROI) ratio (ROI) mean (ROI) ratio (ROI) mean (ROI) ratio (ROI)
10 149 12.41% +87.59% 49 03.63% +96.37% 68 01.26% +98.74%
9 160 13.33% +86.67% 52 03.85% +96.15% 74 01.37% +98.63%
7 192 16.00% +84.00% 60 04.44% +95.56% 87 01.61% +98.39%
5 249 20.75% +79.25% 76 05.63% +94.37% 110 02.04% +97.96%
3 291 24.25% +75.75% 120 08.89% +91.11% 190 03.52% +96.48%
1 621 51.75% +48.25% 273 20.22% +79.78% 1857 34.39% +65.61%
0 1003 83.58% +16.42% 598 44.30% +55.70% 4360 80.74% +19.26%
Max 1200 100% - 1350 100% - 5400 100% -
Ahcen Badji Mokhtar - Annaba University 42 / 91
43. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Energy consumption: results
Per-frame Edetection cost of the method compared to state-of-the-art for size
(240 × 320)3
Method Energy Budget (mJ/Frame)
min (Cyclesdiv = 1) max (Cyclesdiv = 12)
MoG [2] 649.95
CS-MoG [9] 116.44
CoSCS-MoG [10] 125.96
EBSCAM [11] 3.4
FD 0.5069
BIRD (proposed) 0.3723 0.6891
3While we have calculated to best and the worst case, the other techniques have not reported
extreme values. Reported values of other works are shown here.
Ahcen Badji Mokhtar - Annaba University 43 / 91
44. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Limitations / Open challenges
Limitations of the method:
Critical step: threshold value selection.
ROI prioritization requires multi-class classification.
Multi-level classification of the block by its importance is not applied
Solutions
Adaptive and automatic threshold selection resolved in next contribution.
Multi-class ROI classification problem solved in next contribution.
Ahcen Badji Mokhtar - Annaba University 44 / 91
45. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Context and Motivation Model/Method Results and disc
Recap.
We have proposed an energy-efficient ROI detection method for WVS.
The method showed good balance between accuracy, efficiency, and memory
when evaluated on a standard dataset.
The method reduces the processing and compression burden for
resource-constrained surveillance devices.
Next contribution focus on drawback of this method: Multi-class
classification of the ROI, and automatic threshold selection.
Ahcen Badji Mokhtar - Annaba University 45 / 91
46. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 46 / 91
47. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Title: Multi-Threshold-based frame segmentation for content-aware video coding
Book Chapter
Ahcen Badji Mokhtar - Annaba University 47 / 91
48. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Context and Motivation
What is the problem?
Adaptive thresholding for ROI extraction is challenging.
Multi-class region classification based on activity improves encoder process.
ROI resource allocation improves QoS and delivery.
Exploring activity statistics improve classification accuracy.
Needed improvements
Multi-class region classification based on activity (can improves the encoder
process).
ROI resource allocation can improves QoS and delivery.
Exploring activity statistics can improve classification accuracy.
Limits of the SOTA
SOTA ROI detection methods use binary classification.
Fixed threshold is a drawback of SOTA methods.
Accurate adaptive threshold selection in WMSN conditions is challenging.
Solution: What we propose Ahcen Badji Mokhtar - Annaba University 48 / 91
49. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Compress then analyze (CTA) vs. Analyze then
Compress (ATC).
Image acquisition Image Compression Image Transmission
Image Visualization
and Analyze
bitstream
decompression
bitstream reception
Compress-Then-Analyze Paradigm (CTA)
Compress the frame, then analyze it at the destination (CTA).
Image acquisition
ROI-based
Compression
ROI-based
Transmission
Image Visualization
bitstream
deCompression
bitstream reception
ROI Detection and
Analyze
Analyze-Then-Compress Paradigm (ATC)
Analyze the frame, then compress it adaptively (ATC).
Ahcen Badji Mokhtar - Annaba University 49 / 91
50. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Proposed Method
Image acquisition
ROI-based
Compression
ROI-based
Transmission
ROI Detection and
Analyze
Frame n-1 Frame n
Canny Edge Detector
Sum of 8x8 block Absolute
Difference
Fast Gloal Smoother + Maximum
Rank Order Filtering
Automatic Thresholding
Outsu Multi-Threshold
(2 Thresholds )
Activity map masks
First ROI?
set QF=X
set QF=ZY
8x8 block
DCT + Quantization + Huffman coding
yes
no
Buffer
for
ROI 1
bistream
Buffer
for
ROI 2
bitstream
Buffer
for
ROI 2
bitstream
Second ROI?
set QF=YX
yes
no
Proposed multi-class classification of the ROI in an ATC paradigm.
Ahcen Badji Mokhtar - Annaba University 50 / 91
51. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Setup and parameters
Multi-level
Otsu
Thresolding
Used parameters/Techniques for each step
Parameter Value
Edge Detector Canny
SAD 8
FGS
Window size σ
8 0.05
ROF
n p
4 100
Thresholding mult-class Otsu
JPEG
Compression technique Entropic Coding
8-DCT Huffman
Classes QF
X Y
90 50
Ahcen Badji Mokhtar - Annaba University 51 / 91
52. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Classification results
Frame Segmentation for Content-Aware Video Coding in WMSN 7
6: Results of multi-QF based coding. from left to right: 1- Original Frame 2-
mentation Results 3- Decompression results (JPEG chain with ROI1: QF=90,
2:QF=50, ROI3:QF=10 - PSNR=33.9308 , SSIM=0.7618) 4- ROI visual quality
ame bitrate(proposed left, MJPEG right).
sen for comparison due to its low complexity compared to resent encoders and
e it shows large implementation in WMSN. It is shown that the PSNR value
wer for the case of multi-QF in comparison with MJPEG. The reduction is
ROI1: QF=90, ROI2: QF=50, ROI3: QF=10
PSNR=33.9308 dB, SSIM=0.7618
small boxes: ROI visual quality for same bitrate (proposed left, MJPEG right).
Ahcen Badji Mokhtar - Annaba University 52 / 91
53. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Results and discussion: Quality evaluation
0 50 100 150 200 250 300
Frame #
0
10
20
30
40
50
PSNR
[dB]
Proposed (R1-QF=90 R2-QF=50 R3-QF=10
MJPEG
(a) Hall sequence
0 20 40 60 80 100 120
Frame #
0
10
20
30
40
50
PSNR
[dB]
Proposed (R1-QF=90 | R2-QF=50 | R3-QF=10)
MJPEG
(b) Traffic sequence
PSNR value of ROI-based coding compared to MJPEG (at the reception)
The quality of the whole frame marks a degradation of about 9dB compared to
classical method due to used QF values.
Ahcen Badji Mokhtar - Annaba University 53 / 91
54. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Results and discussion: Quality evaluation
MJPEG proposed multi-QF ROI-1
0
10
20
30
40
50
mean
PSNR
[dB]
traffic sequence @bitrate = 1 bpp
(a) Hall sequence
MJPEG proposed multi-QF ROI-1
0
10
20
30
40
50
mean
PSNR
[dB]
hall sequence @bitrate = 1 bpp
(b) Traffic sequence
The mean PSNR of the whole frame of the proposed method, the high priority ROI, and
the whole frame of MJPEG at the same bitrate
The ROI quality is guaranteed, and is higher than non-ROI region and classical
method for a fixed bitrate of 1bpp.
Ahcen Badji Mokhtar - Annaba University 54 / 91
55. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Results and discussion: Bitrate saving
0 50 100 150 200 250 300
Frame #
0
2
4
6
8
Bitrate
(kB)
Proposed (R1-QF=90 R2-QF=50 R3-QF=10
MJPEG(QF=90)
(a) Hall sequence
0 20 40 60 80 100 120
Frame #
0
2
4
6
8
Bitrate
(kB)
Proposed (R1-QF=90 R2-QF=50 R3-QF=10
MJPEG (QF=90)
(b) Traffic sequence
bitrate needed for ROI-based strategy against MJPEG based coding in a Wireless sensor
node
A bit rate gain of almost 50%
Ahcen Badji Mokhtar - Annaba University 55 / 91
56. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Gain and benefits
Discussion
Large reduction of transmission bitrate (generally more than 50%) with respect to
MJPEG..
Reduced bandwidth usage leads to less contention in the channel in WMSN.
For a multi-hop scenarios, energy-constrained nodes relay frames.
For this scenario, the method offers increasing energy savings.
As the number of hops increases, the energy savings increase.
The bit reduction propagates across the network.
Ahcen Badji Mokhtar - Annaba University 56 / 91
57. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Limits and open question
Limits
The proposed method:
Has been tested on limited dataset (3 sequences) / Can be improved in complexity.
Shows relatively large complexity as it consists of many steps.
Has not been evaluated in terms of energy consumption.
solution
The next contribution classifies the frames into multi regions using a novel
addition-based method.
It includes and evaluates detailed energy consumption model.
Ahcen Badji Mokhtar - Annaba University 57 / 91
58. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 58 / 91
59. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Title: Region-of-interest based video coding strategy for rate/energy-constrained
smart surveillance systems using WMSNs: Ad Hoc Networks Journal (Elsevier), IF= 4.9
Ahcen Badji Mokhtar - Annaba University 59 / 91
60. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Methodology: Context and Motivation
Problematic
Reducing the bitrate can affect the quality of the image at the reception.
Enabling some smart tasks at the reception is needed for new networks paradigms.
Better the quality of the ROI, the better the accuracy of the monitoring tasks
Ahcen Badji Mokhtar - Annaba University 60 / 91
61. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Objectives
Reduce power consumption in in-node processing.
Minimize required bandwidth for transmission.
Maintain a high QoS level.
Solution
A novel ROI detector named successive summation of the absolute differences
(S-SAD).
Advantages of the solution:
A tradeoff between quality, bitrate, energy consumption, and object recognition.
An efficient human-based and machine-based smart monitoring tasks.
The method outperforms state-of-the-art and MJPEG techniques using YOLOv3
model.
The energy consumption model confirms the method’s feasibility for IoT nodes.
Ahcen Badji Mokhtar - Annaba University 61 / 91
62. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
ROI Detection
captured
video
Frame Difference
Frame n-1
map1
Sum of 8x8 pixels
Frame n
map2
sum of 4x4 map1
scores
map3
sum of 2x2 map2
scores
1
2
3
4
Thresholding and
blocks labeling
ROI-1
ROI-2
ROI-3
non-ROI
k-1
k
Proposed ROI detection step
Ahcen Badji Mokhtar - Annaba University 62 / 91
63. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Mathematically speaking
SADmap(x, y) =
1
w2
1
w1−1
X
u=0
w1−1
X
v=0
D(w1x + u, w1y + v) (5)
Rmap(x, y) =
1
w2
2
w2−1
X
u=0
w2−1
X
v=0
SADmap(w2x + u, w2y + v) (6)
Gmap(x, y) =
1
w2
3
w3−1
X
u=0
w3−1
X
v=0
Rmap(w3x + u, w3y + v) (7)
ROI-1 ⊂ ROI-2 ⊂ ROI-3
Ahcen Badji Mokhtar - Annaba University 63 / 91
64. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
ROI Detection
Pyramidal view of the calculation and decision
Ahcen Badji Mokhtar - Annaba University 64 / 91
65. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Coding Strategy
Image acquisition ROI-based Compression ROI-based Transmission
ROI Detection and Analyze
Frame k-1 Frame k
Sum of Absolute Differences of
each block (SADmap)
sum of each blocks of
SADmap to get Region Activity
map (Rmap)
sum of each blocks of
Rmap to get Global Activiy map
(Gmap)
Class ?
set
QF=
drop the
block
DCT + Quantization + Huffman coding
yes no
Buffer for
bistream
(high priority)
Buffer for
bitstream
(low priority)
Class ?
set
QF=
yes
no
Thresholding
ROI-1 mask
Thresholding
ROI-2 mask
Thresholding
GMR mask
Global activity map (Gmap)
Succesive Sum
of Absolute
Difference (SSAD)
block
Complete scheme: Coding strategy
Ahcen Badji Mokhtar - Annaba University 65 / 91
66. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Coding Strategy
1 The first priority class C1 = ROI-1 represents the blocks that are
in, and only in the first ROI. Class C1 blocks having the highest interest are
coded with a higher MJPEG quality factor Q1 before being transmitted
2 The second priority class C2 = ROI-2 - ROI-1 includes the labeled moving
blocks that are in ROI-2 but not in ROI-1. Class C2 blocks having a medium
interest are coded, prior to their transmission, with a lower MJPEG quality
factor Q2 Q1
3 The third priority class C3 = GMR - ROI-2 includes the blocks that are
in the ROI but are not in ROI-2. These class blocks are considered to be of
low interest and are simply dropped.
Ahcen Badji Mokhtar - Annaba University 66 / 91
67. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Coding Performances
Performances: assessed for both human-based/machine-based monitoring.
Human-based monitoring: Image quality metrics with and without reference.
Machine-based monitoring: Using Deep Learning model (YOLOv3) for object
recognition.
Ahcen Badji Mokhtar - Annaba University 67 / 91
68. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Results: Visual results Ad Hoc Networks 140 (2023) 1
et al.
Table 2
Visual binary mask for the moving region.
4
4Kouadria et al.:”Region-of-interest based image compression using the discrete Tchebichef
transform in wireless visual sensor networks, 2019.
Ahcen Badji Mokhtar - Annaba University 68 / 91
69. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with Other Methods: Image Quality
Overall mean quality metrics
Sequence
proposed [12] MJPEG
PSNR SSIM VIF PSNR SSIM VIF PSNR SSIM VIF
Highway 31.7414 0.7865 0.6042 30.4667 0.7053 0.5385 32.7808 0.7700 0.7351
HighwayI 28.8923 0.6716 0.5744 31.8053 0.6138 0.4874 37.9583 0.8374 0.7934
HighwayII 28.4600 0.7055 0.4637 29.3400 0.6965 0.4506 33.0100 0.8208 0.7207
campus 31.3614 0.7055 0.4637 29.5200 0.6965 0.4501 35.7400 0.8208 0.7207
intellegentroom 31.7727 0.8036 0.5916 30.4667 0.7053 0.5385 32.7808 0.7700 0.7351
laboratory 32.1748 0.6214 0.5748 30.9583 0.5894 0.5297 34.6275 0.6790 0.7492
Traffic 30.0569 0.6559 0.6230 28.2246 0.5710 0.5030 30.4093 0.6625 0.6493
StreetCornerAtNight 33.3806 0.4686 0.4775 32.1323 0.4402 0.4258 42.4140 0.9034 0.9216
Results
MJPEG performs best for all sequences due to high-quality factor encoding.
Our proposed method achieves second best results for all metrics.
Outperforms MJPEG in SSIM for Intelligentroom and Highway sequences.
Superiority in SSIM is due to stable background with no change over time.
Frames quality decreases for HighwayI and II due to low fps and high motion.
High movement leads to lower quality of ROI-2 (Q2 = 20), decreasing the quality.
Ahcen Badji Mokhtar - Annaba University 69 / 91
70. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with other Methods: PSNR
0 20 40 60 80 100 120
Frame no.
0
10
20
30
40
50
PSNR(dB)
Proposed(50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 100 200 300 400 500 600 700 800
Frame no.
0
10
20
30
40
50
PSNR(dB)
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 50 100 150 200 250 300 350 400
Frame no.
0
10
20
30
40
50
PSNR(dB)
proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
PSNR results for Traffic, Highway and SteertatNight sequences
Results
The quality is very comparable to the standard and better compared to SOTA.
Ahcen Badji Mokhtar - Annaba University 70 / 91
71. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with Other Methods: SSIM
0 20 40 60 80 100 120
Frame no.
0
0.2
0.4
0.6
0.8
1
SSIM
Proposed(50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 100 200 300 400 500 600 700 800
Frame no.
0
0.2
0.4
0.6
0.8
1
SSIM
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 50 100 150 200 250 300 350 400
Frame no.
0
0.2
0.4
0.6
0.8
1
SSIM
proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
SSIM results for Traffic, Highway and SteertatNight sequences
Results
The quality is very comparable to the standard and better than to SOTA.
Ahcen Badji Mokhtar - Annaba University 71 / 91
72. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with other Methods: VIF
0 20 40 60 80 100 120
Frame no.
0
0.2
0.4
0.6
0.8
1
VIF
Proposed(50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 100 200 300 400 500 600 700 800
Frame no.
0
0.2
0.4
0.6
0.8
1
VIF
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 50 100 150 200 250 300 350 400
Frame no.
0
0.2
0.4
0.6
0.8
1
VIF
proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
VIF results for Traffic, Highway and SteertatNight sequences
Results
The quality is very comparable to the standard and better than to SOTA.
Ahcen Badji Mokhtar - Annaba University 72 / 91
73. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with other Methods: BRISQUE
0 20 40 60 80 100 120
Frame no.
15
20
25
30
35
40
45
50
BRISQUE
score
Original
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 100 200 300 400 500 600 700 800
Frame no.
15
20
25
30
35
40
45
BRISQUE
score
Original
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 50 100 150 200 250 300 350 400
Frame no.
25
30
35
40
45
50
55
60
BRISQUE
score
Original
proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
BRISQUE results for Traffic, Highway and SteertatNight sequences
Results
Comparable to the SOTA, the method does not presents no degradation related to no
referential evaluation.
Ahcen Badji Mokhtar - Annaba University 73 / 91
74. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Comparison with other Methods: data size
0 20 40 60 80 100 120
Frame no.
0
0.5
1
1.5
2
2.5
3
data
size
(
kB)
Proposed(QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 100 200 300 400 500 600 700 800
Frame no.
0
2
4
6
8
10
Data
size
(
kB)
Proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
0 50 100 150 200 250 300 350 400
Frame no.
0
2
4
6
8
10
12
Data
size
(
kB)
proposed (QF=50-20)
Kouadria et al. (2019) QF=50
MJPEG QF=50
Requred Data size for Traffic, Highway and SteertatNight sequences
Results
For the campus sequence (fps = 10), our method requires a mean bitrate of 3.358
kB/s, which is 27 times less than the required bitrate with respect to MJPEG
(93.06 kB/s). This represents a saving of 96.4%.
For the highway sequence (fps = 25), we achieve a saving of about 76.3% of the
required bitrate.
Ahcen Badji Mokhtar - Annaba University 74 / 91
75. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Table 4
Bounding box insertion results for the used dataset.
Bounding box insertion results (at the reception)
Video quality remains intact while achieving higher recognition accuracy.
Ahcen Badji Mokhtar - Annaba University 75 / 91
76. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Recognition Accuracy
Traffic Highway HighwayI HighwayII campus intellegentroom laboratory streetCornerAtNight
100
101
102
103
104
Number
of
Recognized
Objects
Original(no compression)
MJPEG QF=50
Kouadria et al. (2019) QF=50
Proposed (QF=50-20)
Mean number of detected objects
Results
Preserving a high quality only for the ROI while ensuring a good ROI detection is
sufficient to enable more accurate smart tasks at the destination like recognition.
Ahcen Badji Mokhtar - Annaba University 76 / 91
77. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Recognition Accuracy
Traffic Highway HighwayI HighwayII campus intellegentroom laboratory streetCornerAtNight
0.5
0.55
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
Recognition
score
Original(no compression)
MJPEG QF=50
Kouadria et al. (2019) QF=50
Proposed (QF=50-20)
Recognition probability
Results
We achieved higher recognition accuracy at lower bitrate and energy budgets, enabling
more accurate smart machine-based tasks (22% enhancement).
Ahcen Badji Mokhtar - Annaba University 77 / 91
78. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Energy consumption: model
Approach
The energy consumption model is computed based on the arithmetic operations
performed in each step.
In our case, the energy model is proportional to the size of the frame and the size
of the window in each step (SAD, Rmap,Gmap) w1 w2 w3.
Eprocessing = Edetection + Ecompression (8)
Edetection = ESADmap + ERmap + EGmap + ET hresh (9)
Ahcen Badji Mokhtar - Annaba University 78 / 91
79. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Energy consumption: Results on ROI detection
Per frame energy cost (mJ) of our ROI detection
Sequence Edetection Eprocessing % extra cost
campus 0.8827 14.24 6.20%
highway 0.6699 16.02 4.18%
traffic 0.1671 20.22 0.83%
Per frame energy consumption (mJ).
Sequence
Proposed MJPEG saving (%)
max min std. dev. mean mean w/r MJPEG
campus 211.14 0.90 24.93 14.24 205.92 93.08
highway 53.38 0.90 10.96 16.02 156 89.74
traffic 40.18 0.23 16.11 20.22 39 48.16
Ahcen Badji Mokhtar - Annaba University 79 / 91
80. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Energy consumption: Results
0 200 400 600 800 1000 1200
Frame no.
0
50
100
150
200
250
Energy
Consumption
(mJ)
27
28
29
30
31
32
33
34
35
36
PSNR(dB)
proposed ROI-based Coding
MJPEG
0 50 100 150 200 250 300
Frame no.
0
20
40
60
80
100
120
140
160
Energy
Consumption
(mJ)
31
32
33
34
35
36
PSNR(dB)
proposed ROI-based Coding
MJPEG
0 20 40 60 80 100
Frame no.
0
10
20
30
40
50
Energy
Consumption
(mJ)
0
10
20
30
40
50
PSNR(dB)
proposed ROI-based Coding
MJPEG
Total processing energy consumption and the corresponding PSNR.
Results analysis
Analysis of energy consumption and quality in terms of PSNR.
Three sequences with frame sizes 352 × 288 ,320 × 240, 160 × 120 are considered.
Energy consumption oscillates based on the size of the ROI.
Our method uses less energy and maintains quality compared to MJPEG (30-35 dB)
The classical method registers a sufficiently stable higher energy consumption value.
Less data processed and sent = less energy consumed.
Ahcen Badji Mokhtar - Annaba University 80 / 91
81. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Limitations and Future Work
Limitations
The channel conditions are not considered here.
The AI-based inference is done only for one task.
Future work
Further study: Further study is recommended to illustrate unusual coding
conditions and issues like the occurrence of outlier frames and/or outlier
blocks during the processing and transmission of the frame.
Ahcen Badji Mokhtar - Annaba University 81 / 91
82. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Model/Method Results and discussion
Recap.
It has been shown that the quality sacrificed of the non-ROI does not
influence the intelligent tasks at the destination but enhances them by virtue
of the content-aware strategy used.
Adopted for large-scale video monitoring: The proposed video coding strategy
could be adopted for large-scale video monitoring in an edge–cloud processing
paradigm using WMSN, where in-network-based scenarios should be
elaborated and assessed.
Ahcen Badji Mokhtar - Annaba University 82 / 91
83. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Outline
1 Introduction
2 Related work
3 Dataset / Setup
4 Region-of-Interest based Video Coding Strategy for Low Bitrate
Surveillance Systems (Contribution 1)
5 An efficient Low Complexity Region-of-Interest Detection for
Video Coding in Wireless Visual Surveillance (Contribution 2)
6 Multi-Threshold-based frame segmentation for content-aware
video coding in WMSN (Contribution 3)
7 Region-of-interest based video coding strategy for rate/energy-
constrained smart surveillance systems using WMSNs
(Contribution 4)
8 General Conclusion
Ahcen Badji Mokhtar - Annaba University 83 / 91
84. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
General Conclusion
We have treated in this thesis the problem of ROI detection and its
implementation as pre-encoder in wireless embedded surveillance systems.
This problem has been studied in the literature and still has many challenges
related to efficiency and accuracy.
We have worked on proposing multiple contributions that develop a pre-encoder
with very low overhead on the total system budget.
The pre-encoder has contributed in the saving of energy and bitrate, achieving
98% of gain.
Ahcen Badji Mokhtar - Annaba University 84 / 91
85. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
General Conclusion
Either the detection efficiency and the gain have been assessed and validated
trough the conducted evaluation strategy and the used dataset/metrics.
The developed system has the capacity to enable easy monitoring for long lifetime
and with acceptable QoS
The developed system has also the capacity to enable both human based
monitoring and Machine-based monitoring opening the door to Cloud Edge based
AI applications for wireless surveillance.
Ahcen Badji Mokhtar - Annaba University 85 / 91
86. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Perspectives
The work can be extended to cover other modules of the wireless sensor node,
especially the used compression algorithm: which can be replaced by fast
transform algorithms.
It can also be extended to cover the adaptation of low-cost transmission protocols
to the context of ROI-based video coding.
After a software validation has been guaranteed trough this thesis, the work can
open the door to an implementation in embedded systems.
Ahcen Badji Mokhtar - Annaba University 86 / 91
87. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Scientific output of the thesis
Peer Reviewed Journal Articles:
JP
[J1] Ahcen Aliouat, Nasreddine Kouadria, Moufida Maimour, Saliha Harize, and
Noureddine Doghmane. ”Region-of-interest based video coding strategy for
rate/ energy-constrained smart surveillance systems using WMSNs.” Ad Hoc
Networks 140 (2023): 103076. IF:4.9
[J2] Ahcen Aliouat, Nasreddine Kouadria, Saliha Harize and Moufida Maimour. ”An
Efficient Low Complexity Region-of-Interest Detection for Video Coding in
Wireless Visual Surveillance.” IEEE Access, 11, 26793-26806. IF: 3.41
[J3] Ahcen Aliouat, Nasreddine Kouadria, Doru Florin Chiper ”x-DTT: A package for
calculating Real and Integer Discrete Tchebichef Transform kernels based on
Orthogonal Polynomials” SoftwareX journal (Minor revision). IF=2.89
[J4] Ahcen Aliouat, Nasreddine Kouadria, Moufida Maimour and Saliha Harize.
”EVBS-CAT: Enhanced Video Background Subtraction with a Controlled
Adaptive Threshold for Constrained Wireless Video-surveillance” Under review:
Journal of Real-Time Image processing (Springer), IF=2.29
Ahcen Badji Mokhtar - Annaba University 87 / 91
88. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Peer-reviewed Conference
Publications/Proceedings
CP
[C1] Ahcen Aliouat, Nasreddine Kouadria, Moufida Maimour, and Saliha Harize.
”Region-of-interest based video coding strategy for low bitrate surveillance
systems.” In 2022 19th International Multi-Conference on Systems, Signals
Devices (SSD), pp. 1357-1362. IEEE, 2022.
[C1] Ahcen Aliouat, Nasreddine Kouadria, Saliha Harize, and Moufida Maimour.
”Multi-threshold-based frame segmentation for content-aware video coding in
WMSN.” In Advances in Computing Systems and Applications: Proceedings of the
5th Conference on Computing Systems and Applications, pp. 337-347. Cham:
Springer International Publishing, 2022.
Ahcen Badji Mokhtar - Annaba University 88 / 91
89. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Poster
Poster
[P1] Ahcen Aliouat, Nasreddine Kouadria and Saliha Harize. ”Low-Cost
Region-of-Interest Detection for Wireless Video Sensor Nodes” In Doctoral Days of
the LASA Laboratory, UBMA, June 2021.
Ahcen Badji Mokhtar - Annaba University 89 / 91
90. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
References
Z. Zivkovic and F. Van Der Heijden, “Efficient adaptive density estimation
per image pixel for the task of background subtraction,” Pattern recognition
letters, vol. 27, no. 7, pp. 773–780, 2006.
C. Stauffer and W. E. L. Grimson, “Adaptive background mixture models for
real-time tracking,” in Proceedings. 1999 IEEE computer society conference
on computer vision and pattern recognition (Cat. No PR00149), vol. 2.
IEEE, 1999, pp. 246–252.
A. Elgammal, D. Harwood, and L. Davis, “Non-parametric model for
background subtraction,” in European conference on computer vision.
Springer, 2000, pp. 751–767.
Y. Benezeth, P.-M. Jodoin, B. Emile, H. Laurent, and C. Rosenberger,
“Comparative study of background subtraction algorithms,” Journal of
Electronic Imaging, vol. 19, no. 3, p. 033003, 2010.
Z. Zivkovic, “Improved adaptive gaussian mixture model for background
subtraction,” in Proceedings of the 17th International Conference on Pattern
Recognition, 2004. ICPR 2004., vol. 2. IEEE, 2004, pp. 28–31.
M. F. Savaş, H. Demirel, and B. Erkal, “Moving object detection using an
Ahcen Badji Mokhtar - Annaba University 90 / 91
91. Introduction Related work Dataset / Setup Region-of-Interest based Video Coding Strategy for Low Bitrate Surveilla
Scientific output of the thesis
Thank You!
Ahcen Badji Mokhtar - Annaba University 91 / 91