Thesis submitted by Andrews Cordolino Sobral at Université de La Rochelle to fulfill the degree of Doctor of Philosophy.
Robust Low-rank and Sparse Decomposition for Moving Object Detection - From Matrices to Tensors
A sharing talk in Hsinchu Coders.
The materials (i.e. images) are from their respective owners:
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
This thesis proposal aims to develop a system called Eureka to efficiently discover training data for visual machine learning tasks. Eureka combines early discard filters, just-in-time machine learning, and the ability to create more accurate filters without writing new code. The goal is to reduce the manual effort required of domain experts to find and label rare phenomena in large unlabeled visual datasets. The proposal outlines research thrusts to apply Eureka in different computing environments like edge, cloud, and smart storage, as well as different problem domains including images, videos, and other multidimensional data. Initial experiments show Eureka can discover more true positives per unit time compared to naive hand-labeling.
(1) The document discusses the Semantic Web, ontologies, and ontology learning. It defines the Semantic Web as an extension of the current web that gives information well-defined meaning. (2) Ontologies are formal specifications of concepts and relations that provide shared meanings between machines and humans. (3) Ontology learning is the automatic or semi-automatic process of extracting ontological concepts and relations from text to build or enrich ontologies. The document outlines methods for ontology learning and its applications.
This document compares the performance of YOLOv3, YOLOv4, and YOLOv5 object detection algorithms for detecting safe landing spots for faulty UAVs using aerial images. It trains the algorithms on the DOTA dataset, which contains images of objects like planes, ships, and sports fields. The algorithms are evaluated on a PC and companion computer based on accuracy metrics like mAP and F1 score as well as inference speed measured in FPS. Preliminary results show YOLOv5 achieves the highest accuracy while maintaining a slightly slower speed than YOLOv4 and YOLOv3. The best performing algorithm will be used for emergency landing spot detection on actual faulty UAVs
Machine Learning Tutorial Part - 1 | Machine Learning Tutorial For Beginners ...Simplilearn
This presentation on Machine Learning will help you understand why Machine Learning came into picture, what is Machine Learning, types of Machine Learning, Machine Learning algorithms with a detailed explanation on linear regression, decision tree & support vector machine and at the end you will also see a use case implementation where we classify whether a recipe is of a cupcake or muffin using SVM algorithm. Machine learning is a core sub-area of artificial intelligence; it enables computers to get into a mode of self-learning without being explicitly programmed. When exposed to new data, these computer programs are enabled to learn, grow, change, and develop by themselves. So, to put simply, the iterative aspect of machine learning is the ability to adapt to new data independently. Now, let us get started with this Machine Learning presentation and understand what it is and why it matters.
Below topics are explained in this Machine Learning presentation:
1. Why Machine Learning?
2. What is Machine Learning?
3. Types of Machine Learning
4. Machine Learning Algorithms
- Linear Regression
- Decision Trees
- Support Vector Machine
5. Use case: Classify whether a recipe is of a cupcake or a muffin using SVM
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars. This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
We recommend this Machine Learning training course for the following professionals in particular:
1. Developers aspiring to be a data scientist or Machine Learning engineer
2. Information architects who want to gain expertise in Machine Learning algorithms
3. Analytics professionals who want to work in Machine Learning or artificial intelligence
4. Graduates looking to build a career in data science and Machine Learning
Learn more at: https://www.simplilearn.com/
Energy-based Model for Out-of-Distribution Detection in Deep Medical Image Se...Seunghyun Hwang
Presented work is accepted in Korean domestic conference, Korean Society of Artificial Intelligence in Medicine (KOSAIM) 2020, as a poster session.
- by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)
This document discusses algorithm-independent machine learning techniques. It introduces concepts like bias and variance, which can quantify how well a learning algorithm matches a problem without depending on a specific algorithm. Methods like cross-validation, bootstrapping, and resampling can be used with different algorithms. While no algorithm is inherently superior, such techniques provide guidance on algorithm use and help integrate multiple classifiers.
Introduction to Named Entity RecognitionTomer Lieber
Named Entity Recognition (NER) is a common task in Natural Language Processing that aims to find and classify named entities in text, such as person names, organizations, and locations, into predefined categories. NER can be used for applications like machine translation, information retrieval, and question answering. Traditional approaches to NER involve feature extraction and training statistical or machine learning models on features, while current state-of-the-art methods use deep learning models like LSTMs combined with word embeddings. NER performance is typically evaluated using the F1 score, which balances precision and recall of named entity detection.
A sharing talk in Hsinchu Coders.
The materials (i.e. images) are from their respective owners:
https://research.googleblog.com/2017/04/federated-learning-collaborative.html
This thesis proposal aims to develop a system called Eureka to efficiently discover training data for visual machine learning tasks. Eureka combines early discard filters, just-in-time machine learning, and the ability to create more accurate filters without writing new code. The goal is to reduce the manual effort required of domain experts to find and label rare phenomena in large unlabeled visual datasets. The proposal outlines research thrusts to apply Eureka in different computing environments like edge, cloud, and smart storage, as well as different problem domains including images, videos, and other multidimensional data. Initial experiments show Eureka can discover more true positives per unit time compared to naive hand-labeling.
(1) The document discusses the Semantic Web, ontologies, and ontology learning. It defines the Semantic Web as an extension of the current web that gives information well-defined meaning. (2) Ontologies are formal specifications of concepts and relations that provide shared meanings between machines and humans. (3) Ontology learning is the automatic or semi-automatic process of extracting ontological concepts and relations from text to build or enrich ontologies. The document outlines methods for ontology learning and its applications.
This document compares the performance of YOLOv3, YOLOv4, and YOLOv5 object detection algorithms for detecting safe landing spots for faulty UAVs using aerial images. It trains the algorithms on the DOTA dataset, which contains images of objects like planes, ships, and sports fields. The algorithms are evaluated on a PC and companion computer based on accuracy metrics like mAP and F1 score as well as inference speed measured in FPS. Preliminary results show YOLOv5 achieves the highest accuracy while maintaining a slightly slower speed than YOLOv4 and YOLOv3. The best performing algorithm will be used for emergency landing spot detection on actual faulty UAVs
Machine Learning Tutorial Part - 1 | Machine Learning Tutorial For Beginners ...Simplilearn
This presentation on Machine Learning will help you understand why Machine Learning came into picture, what is Machine Learning, types of Machine Learning, Machine Learning algorithms with a detailed explanation on linear regression, decision tree & support vector machine and at the end you will also see a use case implementation where we classify whether a recipe is of a cupcake or muffin using SVM algorithm. Machine learning is a core sub-area of artificial intelligence; it enables computers to get into a mode of self-learning without being explicitly programmed. When exposed to new data, these computer programs are enabled to learn, grow, change, and develop by themselves. So, to put simply, the iterative aspect of machine learning is the ability to adapt to new data independently. Now, let us get started with this Machine Learning presentation and understand what it is and why it matters.
Below topics are explained in this Machine Learning presentation:
1. Why Machine Learning?
2. What is Machine Learning?
3. Types of Machine Learning
4. Machine Learning Algorithms
- Linear Regression
- Decision Trees
- Support Vector Machine
5. Use case: Classify whether a recipe is of a cupcake or a muffin using SVM
About Simplilearn Machine Learning course:
A form of artificial intelligence, Machine Learning is revolutionizing the world of computing as well as all people’s digital interactions. Machine Learning powers such innovative automated technologies as recommendation engines, facial recognition, fraud protection and even self-driving cars. This Machine Learning course prepares engineers, data scientists and other professionals with knowledge and hands-on skills required for certification and job competency in Machine Learning.
Why learn Machine Learning?
Machine Learning is taking over the world- and with that, there is a growing need among companies for professionals to know the ins and outs of Machine Learning
The Machine Learning market size is expected to grow from USD 1.03 Billion in 2016 to USD 8.81 Billion by 2022, at a Compound Annual Growth Rate (CAGR) of 44.1% during the forecast period.
We recommend this Machine Learning training course for the following professionals in particular:
1. Developers aspiring to be a data scientist or Machine Learning engineer
2. Information architects who want to gain expertise in Machine Learning algorithms
3. Analytics professionals who want to work in Machine Learning or artificial intelligence
4. Graduates looking to build a career in data science and Machine Learning
Learn more at: https://www.simplilearn.com/
Energy-based Model for Out-of-Distribution Detection in Deep Medical Image Se...Seunghyun Hwang
Presented work is accepted in Korean domestic conference, Korean Society of Artificial Intelligence in Medicine (KOSAIM) 2020, as a poster session.
- by Seunghyun Hwang (Yonsei University, Severance Hospital, Center for Clinical Data Science)
This document discusses algorithm-independent machine learning techniques. It introduces concepts like bias and variance, which can quantify how well a learning algorithm matches a problem without depending on a specific algorithm. Methods like cross-validation, bootstrapping, and resampling can be used with different algorithms. While no algorithm is inherently superior, such techniques provide guidance on algorithm use and help integrate multiple classifiers.
Introduction to Named Entity RecognitionTomer Lieber
Named Entity Recognition (NER) is a common task in Natural Language Processing that aims to find and classify named entities in text, such as person names, organizations, and locations, into predefined categories. NER can be used for applications like machine translation, information retrieval, and question answering. Traditional approaches to NER involve feature extraction and training statistical or machine learning models on features, while current state-of-the-art methods use deep learning models like LSTMs combined with word embeddings. NER performance is typically evaluated using the F1 score, which balances precision and recall of named entity detection.
Machine learning for customer classificationAndrew Barnes
Machine learning can be used to classify customers into segments based on their behavior patterns and perceptions. This allows companies to better target customers. Traditional approaches provide a narrow view of customers due to limited data sources. Machine learning uses both internal customer data and market research to understand customers. Case studies showed how machine learning was used to segment customers for a telecommunications provider to improve bundle fit and satisfaction, segment physicians for a pharmaceutical company to target sales, and predict potential purchasers for a retailer.
Computer vision has received great attention over the last two decades.
This research field is important not only in security-related software, but also in advanced interface between people and computers, advanced control methods and many other areas.
Thesis presentation Slides for Doctorale deplomat obtention of Ph.D. Aliouat Ahcen. Defended on 31-05-2023 in LASA Laboratory, Electronics Department, Faculty of Technology, Badji Mokhtar - Annaba University. Algeria
The presentation title is: Study and Implementation of an Object-based Video Encoder for Embedded Wireless Video Surveillance Systems.
Research Question: How can we detect ROI in a captured video to ensure high-quality encoding and transmission over a WMSN while minimizing bitrate and energy consumption?
Yolo is an end-to-end, real-time object detection system that uses a single convolutional neural network to predict bounding boxes and class probabilities directly from full images. It uses a deeper Darknet-53 backbone network and multi-scale predictions to achieve state-of-the-art accuracy while running faster than other algorithms. Yolo is trained on a merged ImageNet and COCO dataset and predicts bounding boxes using predefined anchor boxes and associated class probabilities at three different scales to localize and classify objects in images with just one pass through the network.
In Comparison with other object detection algorithms, YOLO proposes the use of an end-to-end neural network that makes predictions of bounding boxes and class probabilities all at once.
This document discusses the YOLO object detection algorithm and its applications in real-time object detection. YOLO frames object detection as a regression problem to predict bounding boxes and class probabilities in one pass. It can process images at 30 FPS. The document compares YOLO versions 1-3 and their improvements in small object detection, resolution, and generalization. It describes implementing YOLO with OpenCV and its use in self-driving cars due to its speed and contextual awareness.
1) Deep learning is a type of machine learning that uses neural networks with many layers to learn representations of data with multiple levels of abstraction.
2) Deep learning techniques include unsupervised pretrained networks, convolutional neural networks, recurrent neural networks, and recursive neural networks.
3) The advantages of deep learning include automatic feature extraction from raw data with minimal human effort, and surpassing conventional machine learning algorithms in accuracy across many data types.
Review and discussion of the paper 'Adversarial Attacks and Defenses in Deep Learning' by Kui Ren, Tiahnhang Zheng, Zhan Qin, Xue Liu (2020) for Machine Learning Book Club.
This talk is about how we applied deep learning techinques to achieve state-of-the-art results in various NLP tasks like sentiment analysis and aspect identification, and how we deployed these models at Flipkart
This document summarizes a fall detection system that uses accelerometer data and time series classification algorithms like Dynamic Time Warping (DTW) and K-Nearest Neighbors (KNN) to classify activities into categories like falling, running, walking upstairs, and walking. It reports the accuracy of the classification on test data, achieving an average accuracy of 95% using DTW and 100% accuracy using KNN.
Overview of generative models with the accent to the GANs and deep learning. Includes autoencoders, VAE, normalizing flows, autoregressive models, and a lot of GAN architectures.
Microsoft COCO: Common Objects in Context KhalidKhan412
Datasets are available for facial recognition, action recognition, object detection and recognition, etc. Image datasets are helpful in scene understanding and providing semantic description
This is a deep learning presentation based on Deep Neural Network. It reviews the deep learning concept, related works and specific application areas.It describes a use case scenario of deep learning and highlights the current trends and research issues of deep learning
(1) YOLO frames object detection as a single regression problem to predict bounding boxes and class probabilities directly from full images in one step. (2) It resizes images as input to a convolutional network that outputs a grid of predictions with bounding box coordinates, confidence, and class probabilities. (3) YOLO achieves real-time speeds while maintaining high average precision compared to other detection systems, with most errors coming from inaccurate localization rather than predicting background or other classes.
This document discusses unsupervised learning and clustering algorithms. It begins with an introduction to unsupervised learning, including motivations and differences from supervised learning. It then covers mixture density models, maximum likelihood estimation, and the k-means clustering algorithm. It discusses evaluating clustering using criterion functions and similarity measures. Specific topics covered include normal mixture models, EM algorithm, Euclidean distance, and hierarchical clustering.
Big Data Stockholm v 7 | "Federated Machine Learning for Collaborative and Se...Dataconomy Media
Federated Machine Learning (FedML) is a distributed machine learning approach which enables training on decentralised data. A server coordinates a network of nodes, each of which has local, private training data. The nodes contribute to the construction of a global model by training on local data , and the server combines non-sensitive node model contributions into the global model. Federated learning addresses fundamental problems of centralized AI such as privacy, ownership, and locality of data. It extends, even disrupts, the centralized AI paradigm in which better algorithms always comes at the cost of collecting more and more sensitive data.
1. YOLO proposes a unified object detection model that predicts bounding boxes and class probabilities in one pass of a neural network.
2. It divides the image into a grid and has each grid cell predict B bounding boxes, confidence scores for each box, and C class probabilities.
3. This output is encoded as a tensor and the model is trained end-to-end using a mean squared error between the predicted and true output tensors to optimize localization accuracy and class prediction.
Robust Low-rank and Sparse Decomposition for Moving Object DetectionActiveEon
Presentation summary:
* Moving object detection by background modeling and subtraction.
* Solved and unsolved challenges.
* Framework for low-rank and sparse decomposition.
* Some applications of RPCA on:
* * Background modeling and foreground separation.
* * Very dynamic background.
* * Multidimensional and streaming data.
* LRSLibrary1 + demo.
Invited seminar on "Online Monitoring of Business Constraints and Metaconstraints using LTL and LDL over Finite Traces" given at the University of Luxembourg on January 16, 2015.
Machine learning for customer classificationAndrew Barnes
Machine learning can be used to classify customers into segments based on their behavior patterns and perceptions. This allows companies to better target customers. Traditional approaches provide a narrow view of customers due to limited data sources. Machine learning uses both internal customer data and market research to understand customers. Case studies showed how machine learning was used to segment customers for a telecommunications provider to improve bundle fit and satisfaction, segment physicians for a pharmaceutical company to target sales, and predict potential purchasers for a retailer.
Computer vision has received great attention over the last two decades.
This research field is important not only in security-related software, but also in advanced interface between people and computers, advanced control methods and many other areas.
Thesis presentation Slides for Doctorale deplomat obtention of Ph.D. Aliouat Ahcen. Defended on 31-05-2023 in LASA Laboratory, Electronics Department, Faculty of Technology, Badji Mokhtar - Annaba University. Algeria
The presentation title is: Study and Implementation of an Object-based Video Encoder for Embedded Wireless Video Surveillance Systems.
Research Question: How can we detect ROI in a captured video to ensure high-quality encoding and transmission over a WMSN while minimizing bitrate and energy consumption?
Yolo is an end-to-end, real-time object detection system that uses a single convolutional neural network to predict bounding boxes and class probabilities directly from full images. It uses a deeper Darknet-53 backbone network and multi-scale predictions to achieve state-of-the-art accuracy while running faster than other algorithms. Yolo is trained on a merged ImageNet and COCO dataset and predicts bounding boxes using predefined anchor boxes and associated class probabilities at three different scales to localize and classify objects in images with just one pass through the network.
In Comparison with other object detection algorithms, YOLO proposes the use of an end-to-end neural network that makes predictions of bounding boxes and class probabilities all at once.
This document discusses the YOLO object detection algorithm and its applications in real-time object detection. YOLO frames object detection as a regression problem to predict bounding boxes and class probabilities in one pass. It can process images at 30 FPS. The document compares YOLO versions 1-3 and their improvements in small object detection, resolution, and generalization. It describes implementing YOLO with OpenCV and its use in self-driving cars due to its speed and contextual awareness.
1) Deep learning is a type of machine learning that uses neural networks with many layers to learn representations of data with multiple levels of abstraction.
2) Deep learning techniques include unsupervised pretrained networks, convolutional neural networks, recurrent neural networks, and recursive neural networks.
3) The advantages of deep learning include automatic feature extraction from raw data with minimal human effort, and surpassing conventional machine learning algorithms in accuracy across many data types.
Review and discussion of the paper 'Adversarial Attacks and Defenses in Deep Learning' by Kui Ren, Tiahnhang Zheng, Zhan Qin, Xue Liu (2020) for Machine Learning Book Club.
This talk is about how we applied deep learning techinques to achieve state-of-the-art results in various NLP tasks like sentiment analysis and aspect identification, and how we deployed these models at Flipkart
This document summarizes a fall detection system that uses accelerometer data and time series classification algorithms like Dynamic Time Warping (DTW) and K-Nearest Neighbors (KNN) to classify activities into categories like falling, running, walking upstairs, and walking. It reports the accuracy of the classification on test data, achieving an average accuracy of 95% using DTW and 100% accuracy using KNN.
Overview of generative models with the accent to the GANs and deep learning. Includes autoencoders, VAE, normalizing flows, autoregressive models, and a lot of GAN architectures.
Microsoft COCO: Common Objects in Context KhalidKhan412
Datasets are available for facial recognition, action recognition, object detection and recognition, etc. Image datasets are helpful in scene understanding and providing semantic description
This is a deep learning presentation based on Deep Neural Network. It reviews the deep learning concept, related works and specific application areas.It describes a use case scenario of deep learning and highlights the current trends and research issues of deep learning
(1) YOLO frames object detection as a single regression problem to predict bounding boxes and class probabilities directly from full images in one step. (2) It resizes images as input to a convolutional network that outputs a grid of predictions with bounding box coordinates, confidence, and class probabilities. (3) YOLO achieves real-time speeds while maintaining high average precision compared to other detection systems, with most errors coming from inaccurate localization rather than predicting background or other classes.
This document discusses unsupervised learning and clustering algorithms. It begins with an introduction to unsupervised learning, including motivations and differences from supervised learning. It then covers mixture density models, maximum likelihood estimation, and the k-means clustering algorithm. It discusses evaluating clustering using criterion functions and similarity measures. Specific topics covered include normal mixture models, EM algorithm, Euclidean distance, and hierarchical clustering.
Big Data Stockholm v 7 | "Federated Machine Learning for Collaborative and Se...Dataconomy Media
Federated Machine Learning (FedML) is a distributed machine learning approach which enables training on decentralised data. A server coordinates a network of nodes, each of which has local, private training data. The nodes contribute to the construction of a global model by training on local data , and the server combines non-sensitive node model contributions into the global model. Federated learning addresses fundamental problems of centralized AI such as privacy, ownership, and locality of data. It extends, even disrupts, the centralized AI paradigm in which better algorithms always comes at the cost of collecting more and more sensitive data.
1. YOLO proposes a unified object detection model that predicts bounding boxes and class probabilities in one pass of a neural network.
2. It divides the image into a grid and has each grid cell predict B bounding boxes, confidence scores for each box, and C class probabilities.
3. This output is encoded as a tensor and the model is trained end-to-end using a mean squared error between the predicted and true output tensors to optimize localization accuracy and class prediction.
Robust Low-rank and Sparse Decomposition for Moving Object DetectionActiveEon
Presentation summary:
* Moving object detection by background modeling and subtraction.
* Solved and unsolved challenges.
* Framework for low-rank and sparse decomposition.
* Some applications of RPCA on:
* * Background modeling and foreground separation.
* * Very dynamic background.
* * Multidimensional and streaming data.
* LRSLibrary1 + demo.
Invited seminar on "Online Monitoring of Business Constraints and Metaconstraints using LTL and LDL over Finite Traces" given at the University of Luxembourg on January 16, 2015.
Keynote of HOP-Rec @ RecSys 2018
Presenter: Jheng-Hong Yang
These slides aim to be a complementary material for the short paper: HOP-Rec @ RecSys18. It explains the intuition and some abstract idea behind the descriptions and mathematical symbols by illustrating some plots and figures.
Optimising Autonomous Robot Swarm Parameters for Stable Formation DesignDaniel H. Stolfi
Autonomous robot swarm systems allow to address many inherent limitations of single robot systems, such as scalability and reliability. As a consequence, these have found their way into numerous applications including in the space and aerospace domains like swarm-based asteroid observation or counter-drone systems. However, achieving stable formations around a point of interest using different number of robots and diverse initial conditions can be challenging. In this article we propose a novel method for autonomous robots swarms self-organisation solely relying on their relative position (angle and distance). This work focuses on an evolutionary optimisation approach to calculate the parameters of the swarm, e.g. inter-robot distance, to achieve a reliable formation under different initial conditions. Experiments are conducted using realistic simulations and considering four case studies. The results observed after testing the optimal configurations on 72 unseen scenarios per case study showed the high robustness of our proposal since the desired formation was always achieved. The ability of self-organise around a point of interest maintaining a predefined fixed distance was also validated using real robots.
https://doi.org/10.1145/3512290.3528709
Compositional Blocks for Optimal Self-Healing GradientsRoberto Casadei
This papers revises the state-of-art in gradient computations, provides an evaluation of the performance of different gradient algorithms, presents a new algorithm with multi-path speed optimality, and shows how different techniques and algorithms can be used together to come up with a new optimal gradient implementation.
A Robust Method Based On LOVO Functions For Solving Least Squares ProblemsDawn Cook
The document presents a new robust method for solving least squares problems based on Lower Order-Value Optimization (LOVO) functions. The method combines a Levenberg-Marquardt algorithm adapted for LOVO problems with a voting schema to estimate the number of possible outliers without requiring it as a parameter. Numerical results show the algorithm is able to detect and ignore outliers to find better model fits to data compared to other robust algorithms.
Regression and Classification: An Artificial Neural Network ApproachKhulna University
This presentation introduces artificial neural networks (ANN) as a technique for regression and classification problems. It provides historical context on the development of ANN, describes common network structures and activation functions, and the backpropagation algorithm for training networks. Experimental results on 7 datasets show ANN outperformed other methods for both regression and classification across a variety of problem types and data characteristics. Limitations of ANN and areas for further research are also discussed.
Reapresentação do trabalho na Linux Developer Conference Brazil 2019.
Overview about Linux malware. Extended version including analysis and evasion hands on examples: strace, ltrace, ptrace, ld_preload rootkits.
The document discusses feature extraction and selection for background modeling and foreground detection in videos. It aims to improve background subtraction methods by developing robust visual features. The key contributions are:
1) A novel texture descriptor called eXtended Center-Symmetric Local Binary Pattern (XCS-LBP) which is less sensitive to noise than existing descriptors.
2) Two ensemble learning approaches for feature selection - pixel-based and superpixel-based - to select the optimal feature subsets for different scenes.
3) Experimental results on synthetic video datasets demonstrate the XCS-LBP descriptor outperforms other descriptors for background subtraction under various conditions like noise and weather changes.
The document proposes a model for dynamically organizing edge computing nodes into micro clouds to provide edge computing as a service. The model involves grouping nodes into clusters, clusters into regions, and regions into a topology. Micro clouds are ephemeral cloud-like structures serving local requests before reaching the traditional cloud. Protocols for health checking, cluster formation, and listing the system state are proposed. The model is inspired by cloud architecture and aims to lower latency by processing data closer to its source.
Underwater sparse image classification using deep convolutional neural networksMohamed Elawady
The document discusses using deep convolutional neural networks for underwater sparse image classification. It presents the methodology used, which involves a convolutional neural network framework with hybrid image patching, additional feature maps, and color enhancement techniques. Experimental results on two coral datasets show that the proposed approach achieves good classification accuracy, with some classes classified better than others. The document concludes that this is the first application of deep learning to underwater image processing and introduces a new coral dataset.
SpeQuloS: A QoS Service for BoT Applications Using Best Effort Distributed Co...Gilles Fedak
SpeQuloS: A QoS Service for BoT Applications Using Best Effort Distributed Computing Infrastructures
Simon Delamare Gilles Fedak Derrick Kondo Oleg Lodygensky
High-Performance Parallel and Distributed Computing, 2012
How saccadic models help predict where we look during a visual task? Applicat...Olivier Le Meur
We present saccadic models which are an alternative way to predict where observers look at. Compared to saliency models, saccadic models generate plausible visual scanpaths from which saliency maps can be computed. In addition these models have the advantage of being adaptable to different viewing conditions, viewing tasks and types of visual scene. We demonstrate that saccadic models perform better than existing saliency models for predicting where an observer looks at in free-viewing condition and quality-task condition (i.e. when observers have to score the quality of an image). For that, the joint distributions of saccade amplitudes and orientations in both conditions (i.e. free-viewing and quality task) have been estimated from eye tracking data. Thanks to saccadic models, we hope we will be able to improve upon the performance of saliency-based quality metrics, and more generally the capacity to predict where we look within visual scenes when performing visual tasks.
The document discusses generative design and provides examples of different generative geometry techniques, analysis methods, and automation examples that can be used in a generative design workflow. It introduces key concepts like parametric design, the design space, performance analysis, search techniques, and different types of generative geometry including morphological, data-oriented, rule-based, and behavioral systems. Specific examples are provided for each of these techniques. The generative design workflow is described as consisting of three main modules: generative geometry, analysis, and automation.
This document provides an overview of flexures and compliant mechanisms from a lecture on mechanical design elements. It discusses flexure constraints and degrees of freedom. Examples are given ranging from micro to meso scale precision machines. Material properties important for flexures are outlined. Common flexure modules like hinges, pivots, and bearings are shown and additive rules for combining constraints in series and parallel are covered. Fabrication methods for flexures like EDM, milling, and etching are summarized along with considerations for flexure assembly. The document aims to provide essential information on flexure kinematics, elasticity, design, and manufacturing.
This document summarizes a lecture on flexures and compliant mechanisms. Some key points:
- Flexures allow motion through member compliance and have advantages like smooth motion and miniaturization but disadvantages like limited range of motion and sensitivity to tolerances.
- Important material properties for flexures include modulus, yield stress, and coefficients of thermal expansion.
- Common flexure modules include hinges, levers, ellipses, and cantilevers. Constraint analysis is used to determine degrees of freedom.
- Fabrication methods for flexures include EDM, waterjet cutting, milling, and etching which have tradeoffs in accuracy, speed, and materials used. Proper assembly is also important to minimize hysteresis.
Dynamic scene understanding using temporal association rulesijunejo
This document describes a thesis defense presentation on dynamic scene understanding using temporal association rules. The presentation covers feature extraction using mean-shift tracking, event modeling through spectral clustering of object trajectories, mining frequent temporal patterns and association rules to learn a traffic scene model, and detecting anomalies by comparing test sequences to the learned model. Accuracy of 97% is achieved on junction and roundabout datasets for spatio-temporal anomaly detection.
Double-constrained RPCA based on Saliency Maps for Foreground Detection in Au...ActiveEon
Paper Presentation, ISBC 2015 Workshop conjunction with AVSS 2015, Karlsruhe, Germany, 2015.
Double-constrained RPCA based on Saliency Maps for Foreground Detection in Automated Maritime Surveillance
IRJET- A Real Time Yolo Human Detection in Flood Affected Areas based on Vide...IRJET Journal
This document proposes a method for real-time human detection in flood-affected areas using video content analysis and the YOLO object detection algorithm. It trains YOLO on the COCO Human dataset to detect and localize humans in video frames from surveillance cameras. The results show that YOLO can accurately detect multiple humans, even with occlusion, and single humans under varying illumination. This approach aims to help rescue operations quickly identify affected areas and prioritize aid.
Similar to PhD Thesis Defense Presentation: Robust Low-rank and Sparse Decomposition for Moving Object Detection - From Matrices to Tensors (20)
ENGENHARIA DE COMPUTAÇÃO E INTELIGÊNCIA ARTIFICIALActiveEon
O documento resume a trajetória acadêmica e profissional de Andrews Cordolino Sobral, engenheiro de computação com doutorado em inteligência artificial. Ele possui graduação em engenharia de computação, mestrado focado em processamento de imagens e visão computacional e doutorado em aprendizado de máquina. Atualmente trabalha com pesquisa e desenvolvimento em automação e inteligência artificial.
Machine Learning for Dummies (without mathematics)ActiveEon
It presents an introduction and the basic concepts of machine learning without mathematics. This is a short presentation for beginners in machine learning.
Incremental and Multi-feature Tensor Subspace Learning applied for Background...ActiveEon
ICIAR'14 - International Conference on Image Analysis and Recognition. Incremental and Multi-feature Tensor Subspace Learning applied for Background Modeling and Subtraction.
Comparison of Matrix Completion Algorithms for Background Initialization in V...ActiveEon
Scene Background Modeling and Initialization (SBMI) Workshop in conjunction with ICIAP 2015.
Comparison of Matrix Completion Algorithms for Background Initialization in Videos
Recent advances on low-rank and sparse decomposition for moving object detectionActiveEon
(RFIA 2016) Recent advances on low-rank and sparse decomposition for moving object detection: matrix and tensor-based approaches. RFIA 2016, workshop/atelier: Enjeux dans la détection d’objets mobiles par soustraction de fond.
Online Stochastic Tensor Decomposition for Background Subtraction in Multispe...ActiveEon
Background subtraction is an important task for visual surveillance systems. However, this task becomes more complex when the data size grows since the real-world scenario requires larger data to be processed in a more efficient way, and in some cases, in a continuous manner. Until now, most of background subtraction algorithms were designed for mono or trichromatic cameras within the visible spectrum or near infrared part. Recent advances in multispectral imaging technologies give the possibility to record multispectral videos for video surveillance applications. Due to the specific nature of these data, many of the bands within multispectral images are often strongly correlated. In addition, processing multispectral images with hundreds of bands can be computationally burdensome. In order to address these major difficulties of multispectral imaging for video surveillance, this paper propose an online stochastic framework for tensor decomposition of multispectral video sequences (OSTD). First, the experimental evaluations on synthetic generated data show the robustness of the OSTD with other state of the art approaches then, we apply the same idea on seven multispectral video bands to show that only RGB features are not sufficient to tackle color saturation, illumination variations and shadows problem, but the addition of six visible spectral bands together with one near infra-red spectra provides a better background/foreground separation.
Matrix and Tensor Tools for Computer VisionActiveEon
The document discusses various matrix and tensor tools for computer vision, including principal component analysis (PCA), singular value decomposition (SVD), robust PCA, low-rank representation, non-negative matrix factorization, tensor decompositions, and incremental methods for SVD and tensor learning. It provides definitions and explanations of the techniques along with references for further information.
This document presents a system for classifying traffic patterns using a holistic approach. Video is first processed to estimate crowd density using background subtraction and crowd speed using feature tracking. These measures are used to classify traffic into light, medium or heavy congestion classes. The system was tested on highway traffic videos and achieved 94.5% accuracy using an artificial neural network classifier, comparable to previous work using other classifiers. Misclassifications mostly occurred between medium and heavy congestion patterns.
Classificação Automática do Estado do Trânsito Utilizando Propriedades Holíst...ActiveEon
Este documento resume um trabalho de mestrado sobre análise de tráfego em vídeos utilizando visão computacional. Ele propõe um método holístico que inclui subtração de fundo, rastreamento de veículos e classificação do estado do trânsito. Os algoritmos propostos obtiveram taxas de acerto entre 90-95% na classificação do tráfego.
Unlocking the mysteries of reproduction: Exploring fecundity and gonadosomati...AbdullaAlAsif1
The pygmy halfbeak Dermogenys colletei, is known for its viviparous nature, this presents an intriguing case of relatively low fecundity, raising questions about potential compensatory reproductive strategies employed by this species. Our study delves into the examination of fecundity and the Gonadosomatic Index (GSI) in the Pygmy Halfbeak, D. colletei (Meisner, 2001), an intriguing viviparous fish indigenous to Sarawak, Borneo. We hypothesize that the Pygmy halfbeak, D. colletei, may exhibit unique reproductive adaptations to offset its low fecundity, thus enhancing its survival and fitness. To address this, we conducted a comprehensive study utilizing 28 mature female specimens of D. colletei, carefully measuring fecundity and GSI to shed light on the reproductive adaptations of this species. Our findings reveal that D. colletei indeed exhibits low fecundity, with a mean of 16.76 ± 2.01, and a mean GSI of 12.83 ± 1.27, providing crucial insights into the reproductive mechanisms at play in this species. These results underscore the existence of unique reproductive strategies in D. colletei, enabling its adaptation and persistence in Borneo's diverse aquatic ecosystems, and call for further ecological research to elucidate these mechanisms. This study lends to a better understanding of viviparous fish in Borneo and contributes to the broader field of aquatic ecology, enhancing our knowledge of species adaptations to unique ecological challenges.
The debris of the ‘last major merger’ is dynamically youngSérgio Sacani
The Milky Way’s (MW) inner stellar halo contains an [Fe/H]-rich component with highly eccentric orbits, often referred to as the
‘last major merger.’ Hypotheses for the origin of this component include Gaia-Sausage/Enceladus (GSE), where the progenitor
collided with the MW proto-disc 8–11 Gyr ago, and the Virgo Radial Merger (VRM), where the progenitor collided with the
MW disc within the last 3 Gyr. These two scenarios make different predictions about observable structure in local phase space,
because the morphology of debris depends on how long it has had to phase mix. The recently identified phase-space folds in Gaia
DR3 have positive caustic velocities, making them fundamentally different than the phase-mixed chevrons found in simulations
at late times. Roughly 20 per cent of the stars in the prograde local stellar halo are associated with the observed caustics. Based
on a simple phase-mixing model, the observed number of caustics are consistent with a merger that occurred 1–2 Gyr ago.
We also compare the observed phase-space distribution to FIRE-2 Latte simulations of GSE-like mergers, using a quantitative
measurement of phase mixing (2D causticality). The observed local phase-space distribution best matches the simulated data
1–2 Gyr after collision, and certainly not later than 3 Gyr. This is further evidence that the progenitor of the ‘last major merger’
did not collide with the MW proto-disc at early times, as is thought for the GSE, but instead collided with the MW disc within
the last few Gyr, consistent with the body of work surrounding the VRM.
Immersive Learning That Works: Research Grounding and Paths ForwardLeonel Morgado
We will metaverse into the essence of immersive learning, into its three dimensions and conceptual models. This approach encompasses elements from teaching methodologies to social involvement, through organizational concerns and technologies. Challenging the perception of learning as knowledge transfer, we introduce a 'Uses, Practices & Strategies' model operationalized by the 'Immersive Learning Brain' and ‘Immersion Cube’ frameworks. This approach offers a comprehensive guide through the intricacies of immersive educational experiences and spotlighting research frontiers, along the immersion dimensions of system, narrative, and agency. Our discourse extends to stakeholders beyond the academic sphere, addressing the interests of technologists, instructional designers, and policymakers. We span various contexts, from formal education to organizational transformation to the new horizon of an AI-pervasive society. This keynote aims to unite the iLRN community in a collaborative journey towards a future where immersive learning research and practice coalesce, paving the way for innovative educational research and practice landscapes.
ESR spectroscopy in liquid food and beverages.pptxPRIYANKA PATEL
With increasing population, people need to rely on packaged food stuffs. Packaging of food materials requires the preservation of food. There are various methods for the treatment of food to preserve them and irradiation treatment of food is one of them. It is the most common and the most harmless method for the food preservation as it does not alter the necessary micronutrients of food materials. Although irradiated food doesn’t cause any harm to the human health but still the quality assessment of food is required to provide consumers with necessary information about the food. ESR spectroscopy is the most sophisticated way to investigate the quality of the food and the free radicals induced during the processing of the food. ESR spin trapping technique is useful for the detection of highly unstable radicals in the food. The antioxidant capability of liquid food and beverages in mainly performed by spin trapping technique.
Or: Beyond linear.
Abstract: Equivariant neural networks are neural networks that incorporate symmetries. The nonlinear activation functions in these networks result in interesting nonlinear equivariant maps between simple representations, and motivate the key player of this talk: piecewise linear representation theory.
Disclaimer: No one is perfect, so please mind that there might be mistakes and typos.
dtubbenhauer@gmail.com
Corrected slides: dtubbenhauer.com/talks.html
The use of Nauplii and metanauplii artemia in aquaculture (brine shrimp).pptxMAGOTI ERNEST
Although Artemia has been known to man for centuries, its use as a food for the culture of larval organisms apparently began only in the 1930s, when several investigators found that it made an excellent food for newly hatched fish larvae (Litvinenko et al., 2023). As aquaculture developed in the 1960s and ‘70s, the use of Artemia also became more widespread, due both to its convenience and to its nutritional value for larval organisms (Arenas-Pardo et al., 2024). The fact that Artemia dormant cysts can be stored for long periods in cans, and then used as an off-the-shelf food requiring only 24 h of incubation makes them the most convenient, least labor-intensive, live food available for aquaculture (Sorgeloos & Roubach, 2021). The nutritional value of Artemia, especially for marine organisms, is not constant, but varies both geographically and temporally. During the last decade, however, both the causes of Artemia nutritional variability and methods to improve poorquality Artemia have been identified (Loufi et al., 2024).
Brine shrimp (Artemia spp.) are used in marine aquaculture worldwide. Annually, more than 2,000 metric tons of dry cysts are used for cultivation of fish, crustacean, and shellfish larva. Brine shrimp are important to aquaculture because newly hatched brine shrimp nauplii (larvae) provide a food source for many fish fry (Mozanzadeh et al., 2021). Culture and harvesting of brine shrimp eggs represents another aspect of the aquaculture industry. Nauplii and metanauplii of Artemia, commonly known as brine shrimp, play a crucial role in aquaculture due to their nutritional value and suitability as live feed for many aquatic species, particularly in larval stages (Sorgeloos & Roubach, 2021).
The technology uses reclaimed CO₂ as the dyeing medium in a closed loop process. When pressurized, CO₂ becomes supercritical (SC-CO₂). In this state CO₂ has a very high solvent power, allowing the dye to dissolve easily.
The binding of cosmological structures by massless topological defectsSérgio Sacani
Assuming spherical symmetry and weak field, it is shown that if one solves the Poisson equation or the Einstein field
equations sourced by a topological defect, i.e. a singularity of a very specific form, the result is a localized gravitational
field capable of driving flat rotation (i.e. Keplerian circular orbits at a constant speed for all radii) of test masses on a thin
spherical shell without any underlying mass. Moreover, a large-scale structure which exploits this solution by assembling
concentrically a number of such topological defects can establish a flat stellar or galactic rotation curve, and can also deflect
light in the same manner as an equipotential (isothermal) sphere. Thus, the need for dark matter or modified gravity theory is
mitigated, at least in part.
Authoring a personal GPT for your research and practice: How we created the Q...Leonel Morgado
Thematic analysis in qualitative research is a time-consuming and systematic task, typically done using teams. Team members must ground their activities on common understandings of the major concepts underlying the thematic analysis, and define criteria for its development. However, conceptual misunderstandings, equivocations, and lack of adherence to criteria are challenges to the quality and speed of this process. Given the distributed and uncertain nature of this process, we wondered if the tasks in thematic analysis could be supported by readily available artificial intelligence chatbots. Our early efforts point to potential benefits: not just saving time in the coding process but better adherence to criteria and grounding, by increasing triangulation between humans and artificial intelligence. This tutorial will provide a description and demonstration of the process we followed, as two academic researchers, to develop a custom ChatGPT to assist with qualitative coding in the thematic data analysis process of immersive learning accounts in a survey of the academic literature: QUAL-E Immersive Learning Thematic Analysis Helper. In the hands-on time, participants will try out QUAL-E and develop their ideas for their own qualitative coding ChatGPT. Participants that have the paid ChatGPT Plus subscription can create a draft of their assistants. The organizers will provide course materials and slide deck that participants will be able to utilize to continue development of their custom GPT. The paid subscription to ChatGPT Plus is not required to participate in this workshop, just for trying out personal GPTs during it.
When I was asked to give a companion lecture in support of ‘The Philosophy of Science’ (https://shorturl.at/4pUXz) I decided not to walk through the detail of the many methodologies in order of use. Instead, I chose to employ a long standing, and ongoing, scientific development as an exemplar. And so, I chose the ever evolving story of Thermodynamics as a scientific investigation at its best.
Conducted over a period of >200 years, Thermodynamics R&D, and application, benefitted from the highest levels of professionalism, collaboration, and technical thoroughness. New layers of application, methodology, and practice were made possible by the progressive advance of technology. In turn, this has seen measurement and modelling accuracy continually improved at a micro and macro level.
Perhaps most importantly, Thermodynamics rapidly became a primary tool in the advance of applied science/engineering/technology, spanning micro-tech, to aerospace and cosmology. I can think of no better a story to illustrate the breadth of scientific methodologies and applications at their best.
Travis Hills' Endeavors in Minnesota: Fostering Environmental and Economic Pr...Travis Hills MN
Travis Hills of Minnesota developed a method to convert waste into high-value dry fertilizer, significantly enriching soil quality. By providing farmers with a valuable resource derived from waste, Travis Hills helps enhance farm profitability while promoting environmental stewardship. Travis Hills' sustainable practices lead to cost savings and increased revenue for farmers by improving resource efficiency and reducing waste.
Describing and Interpreting an Immersive Learning Case with the Immersion Cub...Leonel Morgado
Current descriptions of immersive learning cases are often difficult or impossible to compare. This is due to a myriad of different options on what details to include, which aspects are relevant, and on the descriptive approaches employed. Also, these aspects often combine very specific details with more general guidelines or indicate intents and rationales without clarifying their implementation. In this paper we provide a method to describe immersive learning cases that is structured to enable comparisons, yet flexible enough to allow researchers and practitioners to decide which aspects to include. This method leverages a taxonomy that classifies educational aspects at three levels (uses, practices, and strategies) and then utilizes two frameworks, the Immersive Learning Brain and the Immersion Cube, to enable a structured description and interpretation of immersive learning cases. The method is then demonstrated on a published immersive learning case on training for wind turbine maintenance using virtual reality. Applying the method results in a structured artifact, the Immersive Learning Case Sheet, that tags the case with its proximal uses, practices, and strategies, and refines the free text case description to ensure that matching details are included. This contribution is thus a case description method in support of future comparative research of immersive learning cases. We then discuss how the resulting description and interpretation can be leveraged to change immersion learning cases, by enriching them (considering low-effort changes or additions) or innovating (exploring more challenging avenues of transformation). The method holds significant promise to support better-grounded research in immersive learning.
The ability to recreate computational results with minimal effort and actionable metrics provides a solid foundation for scientific research and software development. When people can replicate an analysis at the touch of a button using open-source software, open data, and methods to assess and compare proposals, it significantly eases verification of results, engagement with a diverse range of contributors, and progress. However, we have yet to fully achieve this; there are still many sociotechnical frictions.
Inspired by David Donoho's vision, this talk aims to revisit the three crucial pillars of frictionless reproducibility (data sharing, code sharing, and competitive challenges) with the perspective of deep software variability.
Our observation is that multiple layers — hardware, operating systems, third-party libraries, software versions, input data, compile-time options, and parameters — are subject to variability that exacerbates frictions but is also essential for achieving robust, generalizable results and fostering innovation. I will first review the literature, providing evidence of how the complex variability interactions across these layers affect qualitative and quantitative software properties, thereby complicating the reproduction and replication of scientific studies in various fields.
I will then present some software engineering and AI techniques that can support the strategic exploration of variability spaces. These include the use of abstractions and models (e.g., feature models), sampling strategies (e.g., uniform, random), cost-effective measurements (e.g., incremental build of software configurations), and dimensionality reduction methods (e.g., transfer learning, feature selection, software debloating).
I will finally argue that deep variability is both the problem and solution of frictionless reproducibility, calling the software science community to develop new methods and tools to manage variability and foster reproducibility in software systems.
Exposé invité Journées Nationales du GDR GPL 2024
Deep Software Variability and Frictionless Reproducibility
PhD Thesis Defense Presentation: Robust Low-rank and Sparse Decomposition for Moving Object Detection - From Matrices to Tensors
1. Robust Low-rank and Sparse Decomposition for Moving
Object Detection
From Matrices to Tensors
Andrews Cordolino Sobral
L3I/MIA, Universit´e de La Rochelle
Ph.D. European Label
Supervisors: El-hadi Zahzah (L3I) and Thierry Bouwmans (MIA)
May 11, 2017 - La Rochelle, France
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 1
2. Table of contents
Thesis outline
Introduction to intelligent video surveillance.
Background subtraction process / Challenges.
Framework for low-rank and sparse decomposition / Application for
background/foreground separation.
Contributions:
A unified model for low-rank and sparse decomposition.
Matrix/tensor completion methodology for background model
initialization.
Double constraint RPCA for robust foreground detection in maritime
scenes.
Tensor-based models for handling multidimensional streaming data.
Collaborative external research contributions.
Conclusions and future perspectives.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 2
3. Video surveillance cameras are everywhere
* From IBM Intelligent Video Analytics.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 3
s
4. ...and an intelligent video surveillance system is needed
* From BOSCH Intelligent Video Analysis.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 4
5. ...and an intelligent video surveillance system is needed
* From BOSCH Intelligent Video Analysis.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 5
6. Intrusion detection
Vehicle monitoring
People counting
Abandoned object detection
* From Aventura Intelligent Video Analytics.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 6
7. Understanding an intelligent video surveillance framework
!
Video
Pre-processing
Object
Detection
!
Example of automatic
incident detection
Location of the incident
Intrusion
detection
Abandoned
objects
Tracking and
counting of
people, cars, etc.
Anomaly
detection
Traffic
surveillanceRoad traffic data
collection
Incident report
Human expert
Object Tracking
Activity
Recognition
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 7
8. Our focus
!
Video
Pre-processing
Object
Detection
!
Example of automatic
incident detection
Location of the incident
Intrusion
detection
Abandoned
objects
Tracking and
counting of
people, cars, etc.
Anomaly
detection
Traffic
surveillanceRoad traffic data
collection
Incident report
Human expert
Object Tracking
Activity
Recognition
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 8
9. CCTV systems
The most commonly used equip-
ments are:
Stationary cameras
Pan-Tilt-Zoom (PTZ) cameras
* From Sky NEWS: British Are World’s Most Watched People.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 9
10. Background subtraction (BS) process
Model
initialization
Frames
Model update
Background
model
Foreground
detection
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 10
11. BS challenges
“Solved” and “unsolved” issues:
Baseline
Shadow
Bad weather
Thermal
Dynamic background
Camera jitter
Intermittent object motion
Turbulence
Low framerate
Night scenes
PTZ cameras
* Pierre-Marc Jodoin. Motion Detection: Unsolved Issues
and [Potential] Solutions. Scene Background Modeling and
Initialization (SBMI), ICIAP, 2015.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 11
Play
12. BS methods
A large number of algorithms have been proposed for background
subtraction over the last few years [Sobral and Vacavant, 2014],
[Bouwmans, 2014], [Xu et al., 2016]:
Traditional methods (several implementations in BGSLibrary*):
Basic methods (i.e. [Cucchiara et al., 2001])
Statistical methods (i.e. [Stauffer and Grimson, 1999])
Non-parametric methods (i.e. [Elgammal et al., 2000])
Fuzzy based methods (i.e. [Baf et al., 2008])
Neural and neuro-fuzzy methods (i.e. [Maddalena and Petrosino, 2012])
* [Sobral, 2013] https://github.com/andrewssobral/bgslibrary.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 12
13. BS methods
A large number of algorithms have been proposed for background
subtraction over the last few years [Sobral and Vacavant, 2014],
[Bouwmans, 2014], [Xu et al., 2016]:
Traditional methods (several implementations in BGSLibrary*):
Basic methods (i.e. [Cucchiara et al., 2001])
Statistical methods (i.e. [Stauffer and Grimson, 1999])
Non-parametric methods (i.e. [Elgammal et al., 2000])
Fuzzy based methods (i.e. [Baf et al., 2008])
Neural and neuro-fuzzy methods (i.e. [Maddalena and Petrosino, 2012])
Decomposition into low-rank + sparse components
Introduced in [Cand`es et al., 2011]. In general, the decomposition is done
by matrix and tensor methods. Our focus
* [Sobral, 2013] https://github.com/andrewssobral/bgslibrary.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 13
14. Decomposition into low-rank + sparse components
This framework considers that the data (matrix A) to be processed satisfy
two important assumptions:
The inliers (latent structure) are drawn from a single (or a union of)
low-dimensional subspace(s) (matrix L)
The corruptions are sparse (matrix S)
A L
= +
S
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 14
15. Decomposition into low-rank + sparse components
Note
This assumption holds a particular association to the problem of
background/foreground separation.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 15
16. Decomposition into low-rank + sparse components
Note
This assumption holds a particular association to the problem of
background/foreground separation.
A L
= +
S
The process of background/foreground separation can be regarded as a
matrix separation problem.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 16
17. Robust Principal Component Analysis (RPCA)
This definition is also known as Robust Principal Component Analysis
(RPCA), and can be formulated as follows:
minimize
L,S
rank(L) + card(S),
subject to A = L + S,
(1)
where rank(L) represents the rank of L and card(S) denotes the
number of non-zero entries of S.
The low-rank minimization concerning L offers a suitable framework for
background modeling due to the high correlation between frames.
However, the above equation yields a highly non-convex optimization
problem (NP-hard).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 17
18. RPCA via Principal Component Pursuit (PCP)
[Cand`es et al., 2011] showed that L and S can be recovered by solving a
convex optimization problem, named as Principal Component Pursuit
(PCP).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 18
19. RPCA via Principal Component Pursuit (PCP)
[Cand`es et al., 2011] showed that L and S can be recovered by solving a
convex optimization problem, named as Principal Component Pursuit
(PCP).
The card(.) is replaced with the 1-norm and the rank(.) with the nuclear
norm* ||.||∗, yielding the following convex surrogate:
minimize
L,S
||L||∗ + λ||S||1,
subject to A = L + S,
(2)
where λ > 0 is a trade-off parameter between the sparse and the
low-rank regularization.
The minimization of ||L||∗ enforces low-rankness in L, while the minimization of ||S||1
maximize the sparsity in S.
* Sum of singular values.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 19
20. RPCA limitations
However, the RPCA via PCP has some limitations:
Low-rank component = exactly low-rank.
Sparse component = exactly sparse.
The input matrix is considered as the sum of a true low-rank matrix plus a true sparse
matrix.
That’s not all...
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 20
21. RPCA challenges (outliers)
In real applications the observations are often corrupted by noise, and
missing data can occurs.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 21
22. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 22
23. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 23
24. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Convexity, norms and constraints: Is there a suitable norm or constraint for each
term? Use a convex surrogate norm or not?
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 24
25. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Convexity, norms and constraints: Is there a suitable norm or constraint for each
term? Use a convex surrogate norm or not?
Loss function and regularization: Is there a suitable loss function that is globally
continuous and differentiable? Is there a suitable regularization to improve the
learned model?
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 25
26. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Convexity, norms and constraints: Is there a suitable norm or constraint for each
term? Use a convex surrogate norm or not?
Loss function and regularization: Is there a suitable loss function that is globally
continuous and differentiable? Is there a suitable regularization to improve the
learned model?
Solvers: How to design an efficient optimization algorithm that is faster and more
scalable? online or offline?
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 26
27. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Convexity, norms and constraints: Is there a suitable norm or constraint for each
term? Use a convex surrogate norm or not?
Loss function and regularization: Is there a suitable loss function that is globally
continuous and differentiable? Is there a suitable regularization to improve the
learned model?
Solvers: How to design an efficient optimization algorithm that is faster and more
scalable? online or offline?
Multidimensionality: How to represent the input data?
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 27
28. RPCA challenges (design)
Moreover, designing a RPCA algorithm needs to address some of the
following questions:
Decomposition: Decompose the input data into one, two, or more terms.
Convexity, norms and constraints: Is there a suitable norm or constraint for each
term? Use a convex surrogate norm or not?
Loss function and regularization: Is there a suitable loss function that is globally
continuous and differentiable? Is there a suitable regularization to improve the
learned model?
Solvers: How to design an efficient optimization algorithm that is faster and more
scalable? online or offline?
Multidimensionality: How to represent the input data?
...and without forgetting the BS constraints!
In summary
Designing an efficient RPCA algorithm for background/foreground separation need to
take into account the BS challenges and the mathematical issues of RPCA.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 28
29. RPCA methods
A large number of approaches for robust low-rank and sparse modeling have been
proposed in the last few years ([Zhou et al., 2014], [Lin, 2016],
[Davenport and Romberg, 2016], and [Bouwmans et al., 2016]).
2010–2011 2011–2012 2012–2013 2013–2014 2014–2015 2015–2016
200
400
600
800
1,000
1,200
1,400
# of citations of [Cand`es et al., 2011]*.
* Google Scholar: https://scholar.google.fr/citations?user=nRQi4O8AAAAJ
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 29
30. RPCA methods
A large number of approaches for robust low-rank and sparse modeling have been
proposed in the last few years ([Zhou et al., 2014], [Lin, 2016],
[Davenport and Romberg, 2016], and [Bouwmans et al., 2016]).
2010–2011 2011–2012 2012–2013 2013–2014 2014–2015 2015–2016
200
400
600
800
1,000
1,200
1,400
# of citations of [Cand`es et al., 2011].
In [Bouwmans et al., 2016], more than 300 papers addressed the problem of
background/foreground separation.
Some key issues and challenges remain, such as handling complex/dynamic background
scenarios and performing in a incremental / real-time manner.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 30
32. Decomposition into Low-rank and Sparse Matrices (DLSM)
A unified model is proposed to represent the state-of-the-art methods in
a more general framework, named DLSM (Decomposition into Low-rank
and Sparse Matrices) [Bouwmans, Sobral et al., 2016].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 32
33. Decomposition into Low-rank and Sparse Matrices (DLSM)
A unified model is proposed to represent the state-of-the-art methods in
a more general framework, named DLSM (Decomposition into Low-rank
and Sparse Matrices) [Bouwmans, Sobral et al., 2016].
The DLSM framework categorizes the matrix separation problem into
three main approaches: implicit, explicit and stable.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 33
34. Decomposition into Low-rank and Sparse Matrices (DLSM)
A unified model is proposed to represent the state-of-the-art methods in
a more general framework, named DLSM (Decomposition into Low-rank
and Sparse Matrices) [Bouwmans, Sobral et al., 2016].
The DLSM framework categorizes the matrix separation problem into
three main approaches: implicit, explicit and stable.
and it is formulated as follows:
A =
Y
y=1
Ky (3)
where, in most of the cases, Y ∈ {1, 2, 3}.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 34
35. Implicit approaches: Y = 1
The first matrix K1 is the best low-rank approximation (e.g. K1 = L) of
the matrix A, where A ≈ L.
This is an “implicit decomposition” due to the fact that we have any
constraint with respect to the foreground objects.
The residual matrix S (sparse or not) is recovered by S = A − L.
e.g. Low-Rank Approximation (LRA).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 35
36. Low-Rank Approximation (LRA)
LRA is formulated as:
minimize
L
f (A − L),
subject to rank(L) = r,
(4)
where f (.) denotes a loss function (i.e. ||.||2
F ) and r (1 ≤ r < rank(A)) is the desired rank.
)]kF(. . . vec)1F(vec= [A
kF. . .1FframeskSequence of background modelskSequence of
i
Tviσiu=1i
r
=rA
(rank-1 approximation)1A
Input matrix (full rank) Low-rank approximation
A closed form solution can be estimated by computing the “truncated” Singular Value
Decomposition (SVD) of A.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 36
37. Limitations of LRA
LRA is formulated as:
minimize
L
f (A − L),
subject to rank(L) = r,
(4)
where f (.) denotes a loss function and r (1 ≤ r < rank(A)) represents the desired rank.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 37
38. Affine rank minimization
In many applications, we need to recover a minimal rank matrix subject to
some problem-specific constraints, often characterized as an affine set.
This affine rank minimization problem is defined as follows:
minimize
L
rank(L),
subject to A(L) = b,
(5)
where A : Rm×n → Rp denotes a linear mapping and b ∈ Rp represents a
vector of observations of size p.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 38
39. Matrix Completion (MC)
In many applications, we need to recover a minimal rank matrix subject to
some problem-specific constraints, often characterized as an affine set.
This affine rank minimization problem is defined as follows:
minimize
L
rank(L),
subject to A(L) = b,
(5)
where A : Rm×n → Rp denotes a linear mapping and b ∈ Rp represents a
vector of observations of size p.
A special case of problem (5) is the matrix completion problem:
minimize
L
rank(L),
subject to PΩ(L) = PΩ(A),
(6)
where PΩ(.) denotes a sampling operator restricted to the elements of Ω
(set of observed entries). Let’s take an example!
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 39
40. MC for Background Model Estimation
Conceptual illustration
A ).(ΩPSampling operator )A(ΩP
,)A(ΩP) =L(ΩPsubject to
,∗||L||
L
minimize
L
Application to background estimation
A ).(ΩPSampling operator )A(ΩP
,)A(ΩP) =L(ΩPsubject to
,∗||L||
L
minimize
L
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 40
41. Explicit approaches: Y = 2
The matrices K1 = L and K2 = S are usually assumed to be the low-rank
and sparse representation of the data, where A ≈ L + S.
This is an “explicit decomposition” due to the fact that we have two
constraints: the first one enforcing a low-rank structure over the matrix L,
and the second one enforcing a sparse structure over the matrix S.
Explicit approaches usually work better for the problem of
background/foreground separation in comparison to the implicit methods.
e.g. Robust Principal Component Analysis (RPCA) proposed by [Cand`es et al., 2011].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 41
42. Background/foreground separation with RPCA via PCP
Components
Video Low-rank Sparse Foreground
Background model Moving objects Classification
Demo
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 42
43. Stable approaches: Y = 3
The matrices K1 = L, K2 = S and K3 = E are usually assumed to be the
low-rank, sparse and noise components, respectively, where
A ≈ L + S + E.
This decomposition is called “stable decomposition” as it separates the
sparse components in S and the noise in E.
In the case of background/foreground separation, the noise matrix E can
also represent some dynamic properties of the background.
e.g. Stable Principal Component Pursuit (Stable PCP) proposed by Zhou et
al. [Zhou et al., 2010].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 43
44. PCP vs Stable PCP
Input video RPCA via PCP RPCA via Stable PCP
Visual comparison of foreground segmentation between PCP and Stable PCP for
dynamic background.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 44
45. General overview of the DLSM framework
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 45
46. Focus of the #2 contribution
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 46
48. #2 contribution
Is matrix completion (or even tensor completion) robust to the problem
of background model initialization?
Model
initialization
Frames
Model update
Background
model
Foreground
detection
Let’s see!
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 48
49. Background model (BM) initialization
The main challenge is to obtain a first background model when video
frames contain foreground objects.
Classification of background model initialization methods according to [Bouwmans et al., 2017].
The approaches presented here are in red.
Type of methods Related works
Temporal Statistics Mean, Color Median, MoG [Stauffer and Grimson, 1999], BE-AAPSA [Ramirez-Alonso et al., 2017]
Subintervals of Stable Intensity IMBS-MT [Bloisi et al., 2016], LaBGen [Laugraud et al., 2016]
Model Completion RSL2011 [Reddy et al., 2011]
Optimal Labeling Photomontage [Agarwala et al., 2004]
Subspace Estimation Eigen [Oliver et al., 2000], RSL [De La Torre and Black, 2003], RPCA [Cand`es et al., 2011]
Missing Data Reconstruction Matrix Completion [Sobral et al., 2015a], Tensor Completion [Sobral and Zahzah, 2016]
Neural Networks SC-SOBS [Maddalena and Petrosino, 2012], BEWiS [De Gregorio and Giordano, 2015]
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 49
50. Proposed approach
Low-rank
Reconstruction
original size reduced size
moving pixels
filled with zeros
Motion Detection
Frame Selection
+
background model
input images
Proposed approach to background model initialization.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 50
51. Joint motion detection and frame selection
Frames
0 50 100 150 200 250 300
Differencebetween
consecutiveframes
0
0.2
0.4
0.6
0.8
1
Frame Selection
normalized vector derivative vector selected frames
Illustration of the frame selection operation.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 51
54. Evaluated 13 matrix completion algorithms
List of matrix completion algorithms evaluated for BM initialization.
Type Method Main techniques Author(s)
RM
IALM Augmented Lagrangian [Lin et al., 2010]
RMAMR Augmented Lagrangian [Ye et al., 2015]
MF
SVP Hard thresholding [Jain et al., 2010]
OptSpace Grassmannian [Keshavan et al., 2010]
MC-NMF Non-negative factors [Xu et al., 2012]
LMaFit Alternating [Wen et al., 2012]
ScGrassMC Grassmannian [Ngo and Saad, 2012]
LRGeomCG Riemannian [Vandereycken, 2013]
GROUSE Online algorithm [Balzano and Wright, 2013]
OR1MP Matching pursuit [Wang et al., 2015]
GoDec Randomized [Zhou and Tao, 2011]
SSGoDec Randomized [Zhou and Tao, 2011]
GreGoDec Randomized [Zhou and Tao, 2013]
RM - Rank Minimization
MF - Matrix Factorization
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 54
55. Evaluated 10 tensor completion algorithms algorithms
List of tensor completion algorithms evaluated for BM initialization.
Type Method Main techniques Author(s)
CP
NCPC Non-negative factors [Xu and Yin, 2013]
BCPF Bayesian CP Factorization [Zhao et al., 2015]
TenALS Alternating [Jain and Oh, 2014]
SPC Smooth PARAFAC [Yokota et al., 2016]
TD
HoRPCA-IALM Augmented Lagrangian [Goldfarb and Qin, 2014]
FaLRTC Trace norm [Liu et al., 2013b]
geomCG Riemannian [Kressner et al., 2013]
TMac Alternating [Xu et al., 2015b]
t-SVD Fourier domain [Zhang et al., 2014]
t-TNN Nuclear norm [Hu et al., 2015]
CP - CANDECOMP/PARAFAC decomposition.
TD - Tucker decomposition / HOSVD / N-mode SVD.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 55
56. Dataset
Scene Background Initialization (SBI) dataset
The SBI dataset1 [Maddalena and Petrosino, 2015] contains 14 image sequences and their
corresponding ground truth backgrounds.
1
http://sbmi2015.na.icar.cnr.it/SBIdataset.html
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 56
57. Qualitative results (top-5 algorithms)
Frame
Ground truth
LRGeomCG
LMaFit
RMAMR
MC-NMF
TMac
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 57
58. Quantitative results
Summary of the top-1 best algorithms for each scene.
Scenes Top-1 MC Top-1 TC Scene Top-1
Board IALM TMac M IALM
Candela m1.10 LRGeomCG SPC T SPC
CAVIAR1 LMaFit TMac M LMaFit
CAVIAR2 LRGeomCG TMac M LRGeomCG
CaVignal LRGeomCG TMac M LRGeomCG
Foliage GROUSE TMac M LRGeomCG
HallAndMonitor LRGeomCG t-TNN T t-TNN
HighwayI RMAMR TMac M RMAMR
HighwayII IALM TMac M IALM
HumanBody2 LRGeomCG TMac M LRGeomCG
IBMtest2 LMaFit TMac M LMaFit
PeopleAndFoliage LRGeomCG TMac M LRGeomCG
Snellen LRGeomCG TMac M LRGeomCG
Toscana LRGeomCG SPC M LRGeomCG
M Matrix-based completion.
T Tensor-based completion.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 58
59. Comparison with the state-of-the art
Comparison with the state-of-the art methods [Maddalena and Petrosino, 2015]. The best
scores are in bold, and the top-1 matrix and tensor completion algorithms are highlighted in red
and blue, respectively.
Method AGE pEPs pCEPs MS-SSIM PSNR CQM
Mean 14.1944 22.5150 18.4428 0.8737 25.6980 43.5839
Color Median 10.3744 13.4008 10.5571 0.8533 28.0044 42.4746
MOG2 14.3579 4.0847 2.8080 0.8935 25.9576 38.1916
KNN 20.6968 7.5118 4.5180 0.7595 18.4701 26.3836
BE-AAPSA 11.4846 12.5518 10.0605 0.9247 27.8024 41.8124
WS2006 5.2885 3.5335 1.2118 0.9349 28.8791 39.6334
IMBS-MT 4.2092 3.8819 2.2602 0.9598 33.4090 44.9362
LaBGen 2.9945 1.3972 0.9246 0.9764 35.2028 47.2947
RSL2011 5.8228 5.3511 4.0186 0.9172 29.9272 40.5713
Photomontage 5.8238 4.6952 3.7274 0.9334 31.8573 43.9038
LRGeomCG 8.7644 14.1305 11.0810 0.9302 28.9596 45.5625
TMac 8.8685 14.3577 11.2884 0.9284 28.7507 45.4125
SC-SOBS 1 3.5023 4.1508 2.2295 0.9765 35.2723 50.1138
SC-SOBS 2 4.6049 4.7435 2.5370 0.9645 32.2024 45.7614
BEWIS 3.8665 2.4286 1.4238 0.9675 32.0143 44.3728
http://sbmi2015.na.icar.cnr.it/SBIdataset.html
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 59
60. Remarks
The first four best ranked algorithms (headed by LRGeomCG) are based
on the matrix completion approach.
SBI dataset is based on RGB color images – may not be multidimensional enough for
the power of tensor completion methods.
Tensor-based approaches has been highlighted only on two scenes: Candela m1.10 by
SPC and HallAndMonitor by t-TNN.
Related publications:
(SBMI/ICIAP, 2015, [Sobral et al., 2015a])
(PRL, 2016, [Sobral and Zahzah, 2016]).
MATLAB codes: https://github.com/andrewssobral/mctc4bmi.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 60
61. Focus of the #3 contribution
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 61
62. Dealing with very dynamic background
#3 contribution
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 62
63. Context
The development of automatic video surveillance applications for
maritime environment is a very difficult task due to the complexity of the
scenes: moving water, waves, etc.
The motion of the objects of interest (i.e. ships or boats) can be mixed with the
dynamic behavior of the background (non-regular patterns).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 63
64. Stable PCP for dynamic background scenes
Stable PCP try to deal with this problem under the term where the
multi-modality of the background (i.e. waves) can be considered as
noise component (E).
Some authors used an additional constraint to improve the
background/foreground separation:
[Oreifej et al., 2013] used a turbulence model driven by dense optical
flow to enforce an additional constraint on the rank minimization.
[Ye et al., 2015] proposed a robust motion-assisted matrix restoration
(RMAMR) where a dense motion field given by optical flow is
mapped into a weighting matrix.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 64
65. Proposed method
Combine some ideas of [Oreifej et al., 2013] and [Ye et al., 2015].
The weighting matrix proposed by [Ye et al., 2015] can be used as a
shape constraint (or region constraint),
While the confidence map proposed by [Oreifej et al., 2013]
reinforces the pixels belonging from the moving objects.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 65
66. Proposed method
Combine some ideas of [Oreifej et al., 2013] and [Ye et al., 2015].
The weighting matrix proposed by [Ye et al., 2015] can be used as a
shape constraint (or region constraint),
While the confidence map proposed by [Oreifej et al., 2013]
reinforces the pixels belonging from the moving objects.
Moreover,
Instead of using dense optical flow (temporal descriptor) as a
preliminary step, we suggest to use a saliency detector (spatial
descriptor).
We call our approach as SCM-RPCA (Shape and Confidence Map-based
RPCA).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 66
67. Why a spatial descriptor?
In some cases:
The object of interest can move very slowly (e.g. long distance boats).
The background can be very dynamic (e.g. boats in the sea).
Optical flow may not be sufficient to ensure the object detection.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 67
68. Why a spatial descriptor?
In some cases:
The object of interest can move very slowly (e.g. long distance boats).
The background can be very dynamic (e.g. boats in the sea).
Optical flow may not be sufficient to ensure the object detection.
Moreover,
The dense optical flow computation requests high computational cost, while
computing the saliency map is commonly faster.
Here, the BMS2 method proposed by [Zhang and Sclaroff, 2014] was
selected, due to its speed performance and accuracy results.
2
http://cs-people.bu.edu/jmzhang/BMS/BMS.html
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 68
69. Block diagram of the proposed approach
(a) Input image (b) Saliency detection
(c) Object confidence map
(d) Shape constraint
(e) Foreground mask
RPCA
The double constraints (confidence map and shape) can be built from two different
types of source, but here we focus only on spatial saliency maps.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 69
70. Comparison of the SCM-RPCA and related works.
Author(s) Minimization
Single constraint
[Oreifej et al., 2013] minimize
L,S,E
||L||∗+λ1||Π(S)||1+λ2||E||2
F
subject to A = L + S + E
[Ye et al., 2015] minimize
L,S,E
||L||∗+λ1||S||1+λ2||E||2
F
subject to W ◦ A = W ◦ (L + S + E)
Double constraint
SCM-RPCA (proposed)
[Sobral et al., 2015b]
minimize
L,S,E
||L||∗+λ1||Π(S)||1+λ2||E||2
F
subject to A = L + W ◦ S + E
W weighting matrix / shape constraint (binary case)
Π(.) confidence map
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 70
71. Datasets
UCSD
[Mahadevan and Vasconcelos, 2010]
MarDT
[Bloisi et al., 2013]
The UCSDa and MarDTb datasets consist of 18 and 28 video sequences, respectively, both
acquired from stationary and moving cameras.
a
http://www.svcl.ucsd.edu/projects/background_subtraction/ucsdbgsub_dataset.htm
b
http://www.dis.uniroma1.it/~labrococo/MAR/index.htm
Four sequences from UCSD and three sequences from MarDT were selected, and all sequences
come from stationary cameras.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 71
72. Evaluated algorithms
The SCM-RPCA was compared with its direct competitors:
PCP [Cand`es et al., 2011].
Stable PCP [Aravkin et al., 2014].
3WD [Oreifej et al., 2013]
RMAMR [Ye et al., 2015].
PCP and stable PCP are not constrained, while 3WD and RMAMR are
single-constrained RPCA.
Here, 3WD and RMAMR used saliency maps (instead of optical flow) as
input constraint.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 72
73. Visual comparison over UCSD dataset
Input frame Saliency maps from BMS SCM-RPCA 3WD RMAMRGround truth
surfersboatsbirds
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 73
74. SCM-RPCA over MarDT dataset
Input frame Saliency map Sparse component Foreground mask Ground truthLow-rank component
For the MarDT scenes, the temporal median of the saliency maps was subtracted, due
to the high saliency from the buildings around the river.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 74
76. Quantitative results on UCSD dataset
Quantitative results on four videos of UCSD Background Subtraction Dataset.
Birds Surfers Boats Ocean Rank
Re Pr F1 Re Pr F1 Re Pr F1 Re Pr F1 Avg.F1
PCP 0.842 0.094 0.170 0.754 0.075 0.137 0.814 0.100 0.178 0.748 0.115 0.200 0.171
Lag-SPCP-QN 0.413 0.322 0.362 0.244 0.282 0.261 0.405 0.215 0.281 0.484 0.313 0.380 0.321
RMAMR 0.823 0.229 0.358 0.775 0.248 0.376 0.816 0.230 0.359 0.777 0.175 0.286 0.345
3WD 0.586 0.604 0.595 0.538 0.405 0.462 0.673 0.473 0.556 0.563 0.337 0.422 0.509
SCM-RPCA 0.573 0.638 0.604 0.518 0.565 0.541 0.663 0.550 0.602 0.457 0.544 0.497 0.561
The SCM-RPCA outperformed the previous methods with the highest F-measure
average over the selected video sequences.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 76
77. Computational cost evaluation on UCSD dataset
Computational cost evaluation over four videos of UCSD Background Subtraction Dataset.
Birds Surfers Boats Ocean
(242 × 156 × 71) (344 × 224 × 41) (344 × 224 × 31) (316×196×176)
Iter Time∗
Iter Time∗
Iter Time∗
Iter Time∗
PCP +
100 27.29 +
100 21.19 +
100 18.47 +
100 110.53
Lag-SPCP-QN 29 10.12 53 16.27 39 10.01 18 29.49
RMAMR 34 10.63 35 13.09 33 11.44 35 44.22
3WD 30 4.53 26 4.28 31 4.06 42 29.96
SCM-RPCA 29 4.59 25 4.37 27 3.82 43 33.02
(width × height × length) denotes the frame resolution and the number of processed frames.
∗
Time for matrix decomposition (in seconds). Does not include the time to compute the input constraint (saliency maps).
+
Iteration limit 100 reached.
The algorithms are implemented in MATLAB running on a laptop computer with
Windows 7 Professional 64 bits, 2.7 GHz Core i7-3740QM processor and 32Gb of RAM.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 77
78. Remarks
The experimental results of the SCM-RPCA indicate a better enhancement
of the object foreground mask when compared with its direct competitors.
The combination with confidence map and shape constraint can reduce
the amount of false positive pixels.
The SCM-RPCA algorithm has a slightly change in the number of
iterations and computation time compared to the original 3WD.
Related publication: (IEEE AVSS, 2015, [Sobral et al., 2015b]).
MATLAB codes: https://sites.google.com/site/scmrpca/.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 78
79. Dealing with multidimensional and
streaming data
#4 contribution
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 79
80. Context
Most of background subtraction algorithms were designed for mono (i.e.
graylevel) or trichromatic cameras (i.e. RGB) within the visible spectrum
or near infrared part (NIR).
Recent advances in multispectral imaging technologies give the possibility
to record multispectral videos for video surveillance
applications [Benezeth et al., 2014].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 80
81. Multispectral data
Usually a multispectral video consists of a sequence of multispectral
images sensed from contiguous spectral bands.
Each multispectral image can be represented as a three-dimensional data
cube, or tensor.
Processing a sequence of multispectral images with hundreds of bands
can be computationally expensive.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 81
82. Limitations of matrix-based approaches
Matrix-based low-rank and sparse decomposition methods work only on a
single dimension and consider the input frame as a vector.
Multidimensional data for efficient analysis can not be considered.
The local spatial information is lost and erroneous foreground regions
can be obtained.
Some authors used a tensor representation to solve this
problem [Li et al., 2008, Hu et al., 2011, Tran et al., 2012,
Tan et al., 2013, Sobral et al., 2014, Sobral et al., 2015c].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 82
83. Tensor decomposition and factorization
Tensor decompositions have been widely studied and applied to many
real-world problems [Kolda and Bader, 2009].
They were used to design low-rank approximation algorithms for
multidimensional arrays taking full advantage of the multi-dimensional
structures of the data.
Two widely-used models for low rank decomposition on tensors are:
Tucker/Tucker3 decomposition.
CANDECOMP/PARAFAC (CP) decomposition.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 83
84. Tucker vs CP decomposition
Tucker decomposition
CP decomposition
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 84
85. RPCA on tensors
Recently, some authors extended the Robust PCA framework for matrices
to the multilinear case [Goldfarb and Qin, 2014, Lu et al., 2016].
Tensor Robust PCA decomposition
The RPCA for matrices was reformulated into its “tensorized” version. For
an N-order tensor X, it can be decomposed as:
X = L + S + E, (7)
where L, S and E represent the low-rank, sparse and noise tensors.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 85
86. Proposed approach
Most of tensor subspace learning approaches has the following drawbacks:
Apply matrix SVD into the unfolded matrices (computationally
expensive, especially for large matrices).
Work in a batch manner (not suitable for streaming multispectral
video sequences).
In order to overcome these limitations, we extended the online stochastic
RPCA proposed by [Feng et al., 2013] for tensors.
A stochastic optimization is applied on each mode of the tensor.
The low-dimensional subspace is updated iteratively followed by
processing of one video frame per time instance.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 86
87. Comparison
Stochastic RPCA on matrices [Feng et al., 2013]:
minimize
W,H,S
1
2
||X − WHT
− S||2
F +
λ1
2
(||W||2
F +||H||2
F ) + λ2||S||1,
subject to L = WHT
.
(8)
Extension for tensors (proposed approach) [Sobral et al., 2015c]:
minimize
W,H,S
1
2
N
i=1
||X[i]
− Wi HT
i − S[i]
||2
F +
λ1
2
(||Wi ||2
F +||Hi ||2
F ) + λ2||S[i]
||1,
subject to L[i]
= Wi HT
i .
(9)
X[n]
: n-mode matricization of tensor X.
Xi : ith matrix.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 87
88. Dataset
MVS (Multispectral Video Sequences) dataset [Benezeth et al., 2014]
The proposed method was evaluated on MVS dataset. This is the first dataset on MVSa
available for research community in background subtraction.
a
http://ilt.u-bourgogne.fr/benezeth/projects/ICRA2014/
The MVS dataset contains a set of 5 video sequences with 7 multispectral bands (6 visible
spectra and 1 NIR spectrum). Each sequence presents a well known BS challenge, such as color
saturation and dynamic background.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 88
89. Evaluated algorithms
The proposed approach was compared with 3 other ones:
CP-ALS [Kolda and Bader, 2009]
HORPCA [Goldfarb and Qin, 2014]
BRTF [Zhao et al., 2016]
CP-ALS, HORPCA, and BRTF are based on batch optimization strategy.
Due to this limitation, they were applied for each 100 frames at time
(reducing the computational cost) of the whole video sequence
(fourth-order tensor).
The proposed method processes each multispectral image or RGB image
per time instance.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 89
90. Qualitative results I
RGB image ground truth proposed approach BRTF HORPCA CP-ALS
Visual comparison of background subtraction results over three scenes of
the MVS dataset.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 90
92. Qualitative results III
Input
Low-rank
Sparse
Mask
RGB VS-1 VS-2 VS-3 VS-4 VS-5 VS-6 NIR
Visual results of the proposed method on each RGB and multispectral
band.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 92
94. Computational time
Computational time for the first 100 frames varying the image resolution.
Size HORPCA CP-ALS BRTF Proposed
160 × 120 00:01:35 00:00:40 00:00:22 00:00:04
320 × 240 00:04:56 00:02:09 00:03:50 00:00:12
The algorithms were implemented in MATLAB running on a laptop
computer with Windows 7 Professional 64 bits, 2.7 GHz Core i7-3740QM
processor and 32Gb of RAM.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 94
95. Remarks
Experimental results show that the proposed methodology outperforms the
other considered approaches.
We have achieved almost real time processing, since one video frame is
processed at time.
Related publications:
(ICIAR, 2014, [Sobral et al., 2014]).
(IEEE ICCV Workshop on RSL-CV, 2015, [Sobral et al., 2015c]).
MATLAB codes: https://github.com/andrewssobral/ostd.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 95
97. Collaborative research with Jordi Gonzalez at CVC
(Barcelona, Spain)
Evaluation of subspace clustering algorithms to the problem of
human action recognition from 3D skeletal data (work in progress).
Robust subspace clustering of human activities through skeletal data.
Differently from previous approaches, subspace clustering methods consider
the inliers are drawn from the union of low-dimensional subspaces.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 97
98. Building the actions representation matrix
Construction of the action representation matrix.
Temporal modeling procedure applied in the skeletal representation to deal with rate variations,
temporal misalignment, and noise.
* From [Vemulapalli et al., 2014].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 98
99. Setup
Datasets for human action recognition from 3D skeletal data.
Dataset # of actions # of subjects # of sequences
UTKinect-Action [Xia et al., 2012]3 10 10 199
Florence3D-Action [Seidenari et al., 2013]4 9 10 215
Skeletal representations:
AJP (Absolute Joint Positions).
RJP (Relative Joint Positions).
JAQ (Joint Angles Quaternions).
SE3AP (SE3 Lie Algebra with Absolute Pairs) [Vemulapalli et al., 2014].
SE3RP (SE3 Lie Algebra with Relative Pairs) [Vemulapalli et al., 2014].
3
http://cvrc.ece.utexas.edu/KinectDatasets/HOJ3D.html
4
http://www.micc.unifi.it/vim/datasets/3dactions/
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 99
100. Evaluated algorithms
Selected subspace clustering algorithms for evaluation on skeletal action datasets.
Representation Method Author(s)
low-rank
LRR [Liu et al., 2013a]
LRSC [Vidal and Favaro, 2014]
sparse
SSC [Elhamifar and Vidal, 2009]
RSSC [Xu et al., 2015a]
LS3C [Patel et al., 2013]
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 100
101. Preliminary results (work in progress)
Performance comparison with state-of-the-art methods.
Author(s) Approach Recognition rate
UTKinect-Action dataset
[Xia et al., 2012] Histograms of 3D joints 90.92%
[Zhu et al., 2013] Random forests 87.90%
[Vemulapalli et al., 2014] Points in a Lie Group 97.08%
proposed LRSC + AJP or RSSC + RJP 95.10%
Florence3D-Action dataset
[Seidenari et al., 2013] Multi-Part Bag-of-Poses 82.00%
[Cippitelli et al., 2016] Key poses 82.10%
[Vemulapalli et al., 2014] Points in a Lie Group 90.88%
proposed RSSC + AJP 79.00%
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 101
103. Hierarchical overview of the DLSM framework
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 103
104. Summary and contributions
The thesis presented here has provided the following contributions:
A unified model for low-rank and sparse decomposition.
Matrix/tensor completion methodology for background model
initialization.
Double-constrained version of RPCA for robust foreground detection
in dynamic background.
Tensor-based methods for background/foreground separation in
multidimensional streaming data.
A collaborative work in conjunction with CVC/UAB that enabled the
European Label of this thesis, and a publication project.
Finally, a new library, named LRSLibrary, that provides a collection of
low-rank and sparse decomposition algorithms.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 104
105. Future perspectives
Matrix/tensor completion methodology
More robust approach for frame-selection.
Evaluation of incremental and real-time approaches.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 105
106. Future perspectives
Matrix/tensor completion methodology
More robust approach for frame-selection.
Evaluation of incremental and real-time approaches.
SCM-RPCA
How different sources can improve the foreground segmentation.
Development of an incremental version for streaming applications.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 106
107. Future perspectives
Matrix/tensor completion methodology
More robust approach for frame-selection.
Evaluation of incremental and real-time approaches.
SCM-RPCA
How different sources can improve the foreground segmentation.
Development of an incremental version for streaming applications.
Tensor-based methods
Consider the recent advances on randomized
RPCA [Erichson et al., 2016].
Implementation C/C++ with GPU support for high scalability.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 107
108. Publications I
The thesis has led to the following publications5:
Talks (1)
2016 - Sobral, Andrews. “Recent advances on low-rank and sparse decomposition for
moving object detection.”. Workshop/atelier: Enjeux dans la d´etection d’objets mobiles
par soustraction de fond. Reconnaissance de Formes et Intelligence Artificielle (RFIA),
20166.
Journal papers (4)
2017 - Sobral, Andrews; Gong, Wenjuan; Gonzalez, Jordi; Bouwmans, Thierry; Zahzah,
El-hadi. “Robust Subspace Clustering of Human Activities from 3D Skeletal Data”, (in
progress).
2016 - Sobral, Andrews; Zahzah, El-hadi. “Matrix and Tensor Completion Algorithms for
Background Model Initialization: A Comparative Evaluation”, In the Special Issue on
Scene Background Modeling and Initialization (SBMI), Pattern Recognition Letters
(PRL), 2016. [Sobral and Zahzah, 2016].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 108
109. Publications II
2016 - Gong, Wenjuan; Zhang, Xuena; Gonzalez, Jordi; Sobral, Andrews; Bouwmans,
Thierry; Tu, Changhe; Zahzah, El-hadi. “Human Pose Estimation from Monocular
Images: A Comprehensive Survey”, Sensors, 2016. [Gong et al., 2016].
2016 - Bouwmans, Thierry; Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Zahzah,
El-Hadi. “Decomposition into Low-rank plus Additive Matrices for
Background/Foreground Separation: A Review for a Comparative Evaluation with a
Large-Scale Dataset”, Computer Science Review, 2016. [Bouwmans et al., 2016].
Books (1)
2017 - Bouwmans, Thierry; Sobral, Andrews; Zahzah, El-hadi. Handbook on
“Background Subtraction for Moving Object Detection: Theory and Practices”, (in
progress)7.
Book chapters (2)
2017 - Sobral, Andrews; Bouwmans, Thierry; Zahzah, El-hadi. “Robust Tensor Models”.
Chapter in the handbook “Background Subtraction for Moving Object Detection: Theory
and Practices”, (in progress).
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 109
110. Publications III
2015 - Sobral, Andrews; Bouwmans, Thierry; Zahzah, El-hadi. “LRSLibrary: Low-Rank
and Sparse tools for Background Modeling and Subtraction in Videos”. Chapter in the
handbook “Robust Low-Rank and Sparse Matrix Decomposition: Applications in Image
and Video Processing”, CRC Press, Taylor and Francis Group, 2015. [Sobral et al., 2016].
Conferences (7)
2015 - Sobral, Andrews; Javed, Sajid; Ki Jung, Soon; Bouwmans, Thierry; Zahzah,
El-hadi. “Online Stochastic Tensor Decomposition for Background Subtraction in
Multispectral Video Sequences”. ICCV Workshop on Robust Subspace Learning and
Computer Vision (RSL-CV), Santiago, Chile, December, 2015. [Sobral et al., 2015c].
2015 - Javed, Sajid; Ho Oh, Seon; Sobral, Andrews; Bouwmans, Thierry; Ki Jung, Soon.
“Background Subtraction via Superpixel-based Online Matrix Decomposition with
Structured Foreground Constraints”. ICCV Workshop on Robust Subspace Learning and
Computer Vision (RSL-CV), Santiago, Chile, December, 2015. [Javed et al., 2015a].
2015 - Sobral, Andrews; Bouwmans, Thierry; Zahzah, El-hadi. ”Comparison of Matrix
Completion Algorithms for Background Initialization in Videos”. Scene Background
Modeling and Initialization (SBMI), Workshop in conjunction with ICIAP 2015, Genova,
Italy, September, 2015. [Sobral et al., 2015a].
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 110
111. Publications IV
2015 - Sobral, Andrews; Bouwmans, Thierry; Zahzah, El-hadi. “Double-constrained
RPCA based on Saliency Maps for Foreground Detection in Automated Maritime
Surveillance”. Identification and Surveillance for Border Control (ISBC), International
Workshop in conjunction with AVSS 2015, Karlsruhe, Germany, August,
2015. [Sobral et al., 2015b].
2015 - Javed, Sajid; Sobral, Andrews; Bouwmans, Thierry; Ki Jung, Soon. “OR-PCA
with Dynamic Feature Selection for Robust Background Subtraction”. In Proceedings of
the 30th ACM/SIGAPP Symposium on Applied Computing (ACM-SAC), Salamanca,
Spain, 2015. [Javed et al., 2015b].
2014 - Javed, Sajid; Ho Oh, Seon; Sobral, Andrews; Bouwmans, Thierry; Ki Jung, Soon.
“OR-PCA with MRF for Robust Foreground Detection in Highly Dynamic Backgrounds”.
In the 12th Asian Conference on Computer Vision (ACCV 2014), Singapore, November,
2014. [Javed et al., 2014].
2014 - Sobral, Andrews; Baker, Christopher G.; Bouwmans, Thierry; Zahzah, El-hadi.
“Incremental and Multi-feature Tensor Subspace Learning applied for Background
Modeling and Subtraction”. International Conference on Image Analysis and Recognition
(ICIAR’2014), Vilamoura, Algarve, Portugal, October, 2014. [Sobral et al., 2014].
5
The reader can refer to https://scholar.google.fr/citations?user=0Nm0uHcAAAAJ for an updated list of publications
and their citations.
6
http://rfia2016.iut-auvergne.com/index.php/autres-evenements/
detection-d-objets-mobiles-par-soustraction-de-fond
7
https://sites.google.com/site/foregrounddetection/
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 111
113. LRSLibrary
A new library, named LRSLibrary [Sobral et al., 2016]a
that provides a collection of
low-rank and sparse decomposition algorithms in MATLAB.
a
https://github.com/andrewssobral/lrslibrary
The LRSLibrary was designed for background/foreground separation in videos, and it
contains a total of 104 matrix-based and tensor-based algorithms.
It has been fundamental for all the experiments conducted in the thesis.
Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 113
114. [Agarwala et al., 2004] Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and
Cohen, M. (2004). Interactive digital photomontage. In ACM SIGGRAPH, SIGGRAPH ’04, pages 294–302, New York, NY,
USA. ACM.
[Aravkin et al., 2014] Aravkin, A. Y., Becker, S., Cevher, V., and Olsen, P. (2014). A variational approach to stable principal
component pursuit. The Conference on Uncertainty in Artificial Intelligence.
[Baf et al., 2008] Baf, F. E., Bouwmans, T., and Vachon, B. (2008). Fuzzy integral for moving object detection. In IEEE
International Conference on Fuzzy Systems, pages 1729–1736.
[Balzano and Wright, 2013] Balzano, L. and Wright, S. J. (2013). On GROUSE and incremental SVD. In IEEE International
Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).
[Benezeth et al., 2014] Benezeth, Y., Sidibe, D., and Thomas, J. B. (2014). Background subtraction with multispectral video
sequences. In International Conference on Robotics and Automation (ICRA).
[Bloisi et al., 2013] Bloisi, D. D., Iocchi, L., and Pennisi, A. (2013). Mar - maritime activity recognition dataset.
[Bloisi et al., 2016] Bloisi, D. D., Pennisi, A., and Iocchi, L. (2016). Parallel multi-modal background modeling. Pattern
Recognition Letters, pages –.
[Bouwmans, 2014] Bouwmans, T. (2014). Traditional and recent approaches in background modeling for foreground detection:
An overview. In Computer Science Review.
[Bouwmans et al., 2017] Bouwmans, T., Maddalena, L., and Petrosino, A. (2017). Scene background initialization: a
taxonomy. Pattern Recognition Letters.
[Bouwmans et al., 2016] Bouwmans, T., Sobral, A., Javed, S., Jung, S. K., and Zahzah, E. (2016). Decomposition into
low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a
large-scale dataset. Computer Science Review.
[Cand`es et al., 2011] Cand`es, E. J., Li, X., Ma, Y., and Wright, J. (2011). Robust Principal Component Analysis? Journal of
the ACM.
[Chang et al., 2015] Chang, X., Nie, F., Ma, Z., Yang, Y., and Zhou, X. (2015). A convex formulation for spectral shrunk
clustering. In AAAI Conference on Artificial Intelligence.
[Cippitelli et al., 2016] Cippitelli, E., Gasparrini, S., Gambi, E., and Spinsante, S. (2016). A human activity recognition system
using skeleton data from rgbd sensors. Journal of Computational Intelligence and Neuroscience.Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 114
115. [Agarwala et al., 2004] Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and
Cohen, M. (2004). Interactive digital photomontage. In ACM SIGGRAPH, SIGGRAPH ’04, pages 294–302, New York, NY,
USA. ACM.
[Aravkin et al., 2014] Aravkin, A. Y., Becker, S., Cevher, V., and Olsen, P. (2014). A variational approach to stable principal
component pursuit. The Conference on Uncertainty in Artificial Intelligence.
[Baf et al., 2008] Baf, F. E., Bouwmans, T., and Vachon, B. (2008). Fuzzy integral for moving object detection. In IEEE
International Conference on Fuzzy Systems, pages 1729–1736.
[Balzano and Wright, 2013] Balzano, L. and Wright, S. J. (2013). On GROUSE and incremental SVD. In IEEE International
Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP).
[Benezeth et al., 2014] Benezeth, Y., Sidibe, D., and Thomas, J. B. (2014). Background subtraction with multispectral video
sequences. In International Conference on Robotics and Automation (ICRA).
[Bloisi et al., 2013] Bloisi, D. D., Iocchi, L., and Pennisi, A. (2013). Mar - maritime activity recognition dataset.
[Bloisi et al., 2016] Bloisi, D. D., Pennisi, A., and Iocchi, L. (2016). Parallel multi-modal background modeling. Pattern
Recognition Letters, pages –.
[Bouwmans, 2014] Bouwmans, T. (2014). Traditional and recent approaches in background modeling for foreground detection:
An overview. In Computer Science Review.
[Bouwmans et al., 2017] Bouwmans, T., Maddalena, L., and Petrosino, A. (2017). Scene background initialization: a
taxonomy. Pattern Recognition Letters.
[Bouwmans et al., 2016] Bouwmans, T., Sobral, A., Javed, S., Jung, S. K., and Zahzah, E. (2016). Decomposition into
low-rank plus additive matrices for background/foreground separation: A review for a comparative evaluation with a
large-scale dataset. Computer Science Review.
[Cand`es et al., 2011] Cand`es, E. J., Li, X., Ma, Y., and Wright, J. (2011). Robust Principal Component Analysis? Journal of
the ACM.
[Chang et al., 2015] Chang, X., Nie, F., Ma, Z., Yang, Y., and Zhou, X. (2015). A convex formulation for spectral shrunk
clustering. In AAAI Conference on Artificial Intelligence.
[Cippitelli et al., 2016] Cippitelli, E., Gasparrini, S., Gambi, E., and Spinsante, S. (2016). A human activity recognition system
using skeleton data from rgbd sensors. Journal of Computational Intelligence and Neuroscience.Andrews Cordolino Sobral (L3I/MIA) Universit´e de La Rochelle 115