Successfully reported this slideshow.
Your SlideShare is downloading. ×

Survey on Evolutionary Computation of Computer Vision

Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad
Ad

Check these out next

1 of 37 Ad
Advertisement

More Related Content

Recently uploaded (20)

Advertisement

Survey on Evolutionary Computation of Computer Vision

  1. 1. A SURVEY ON EVOLUTIONARY COMPUTATION FOR COMPUTER VISION AND IMAGE ANALYSIS. Past, Present, and Future Trends - Nisha Mohan Devadiga
  2. 2. INTRODUCTION • In Computer Vision (CV), one of the most important tasks is image analysis, which aims to extract and analyze meaningful information from images. • Evolutionary computation (EC) techniques are evolutionary algorithms, swarm intelligence and others. • Many problems in engineering, management, business ,, remote sensing, biology, finance, manufacturing, and medicine have been successfully solved using EC techniques. • EC solves functions/parameter optimization, classification, regression, and clustering, among other optimization and learning-related problems.
  3. 3. ADVANTAGES • ECs that can be used for a lot of CV and image analysis tasks and are effective without requiring a lot of domain knowledge. • By producing a set of non-dominated solutions, EC techniques are well suited for solving problems with multiple competing goals. Multi-objective optimization problems are those that are difficult to solve with exact techniques. • EC methods have been effectively applied to CV and image analysis and hence such area of research is known as ECV, evolutionary computer vision.
  4. 4. TAXONOMY AND SCOPE EC for Images Analysis Task Type Edge detection Image segmentation Image feature analysis Object detection Image classification Others Solution Representation Fixed length Variable length Method Type Evolutionary algorithms Swarm Intelligence No of Objective Single- objective methods Multi- objective methods Role of the method Optimization in specific solutions Learning solution from scratch Application Domain Facial images Biomedical images Satellite images Others Fig 1. Categories of existing EC methods for image analysis. Image Source - https://ieeexplore.ieee.org/document/9943992
  5. 5. EDGE DETECTION • Finding or detecting discontinuities in the image's pixel values is the goal of edge detection. • To approximate the pixel value gradient, a straightforward solution is to compare the value of the current pixel to the values of its neighboring pixels from various directions. The current pixel may become an edge pixel if it significantly changes. • Utilized in image segmentation and object detection. However, edge detection is difficult because of the noise and complex background in images. As can be seen in figure, EC methods have been utilized extensively for edge detection. Image Source - https://ieeexplore.ieee.org/document/9943992
  6. 6. EDGE DETECTION - ANT COLONY OPTIMIZATION (ACO) • The most popular approach for graph-based edge detection is "Ant Colony Optimization (ACO)." • A graph in which each node represents a pixel can be used to represent an image. Ants can move from one node to another and mark each node by increasing the cell in a "pheromone" matrix that corresponds to it. • By imposing a threshold on the ant pheromone, edges can be identified. • The primary operations of the ACO edge detection techniques are the ant movement from one node to another and the update of the pheromone matrix. • Image enhancement, filtering, and edge detection are the three phases of the edge detection procedure.
  7. 7. EDGE DETECTION - ANT COLONY OPTIMIZATION (ACO) Benefits:- • ACO is used to locate the image's edge points. • During the search, each ant can move to four adjacent directions. • Additionally, ACO has been utilized to enhance the outcomes of conventional edge detection techniques. • Two conventional edge detectors' results can be improved by ACO by repairing broken edges. • In terms of improving edge detection performance, it is both efficient and effective.
  8. 8. EDGE DETECTION - GENETIC PROGRAMMING (GP) • "Genetic Programming" (GP) models that automatically categorize pixels into non- edge or edge groups or identify edge pixels from images are built. • W. Fu and co. in order to create natural image-based tree-based edge detectors, create a new GP method with a new function set and program structure. • Based on a predetermined threshold, the evolved edge detectors classify a pixel as an edge or non-edge point. The outcomes demonstrate that the leaf nodes in the edge detectors are crucial and that the automatically constructed edge detectors are very impressive.
  9. 9. EDGE DETECTION - OTHER EC METHODS • Other EC methods are used to optimize edge detection process. • The optimal coefficients of the cloning template in a cellular neural network for edge detection can be found using Differential Evolution (DE). • As inputs to the generator of the generative adversarial network (GAN) for edge detection, an improved DE method is proposed for producing better images. • The loss function derived from the GAN's discriminator is DE's fitness function. On two benchmark datasets, this method performs at the cutting edge. However, it takes a long time and several real images to train the models. • In conclusion, EC techniques can automatically evolve edge detectors, construct graphs for edge detection, optimize traditional detector results, and optimize existing algorithm parameters when used for edge detection.
  10. 10. IMAGE SEGMENTATION • The goal of image segmentation is to split an image into multiple uniform and uncorrelated regions. • It is a crucial step in image analysis, and it is frequently required to complete higher-level tasks like object detection and image classification. • It is a challenging endeavor that may necessitate extensive domain knowledge, high computational costs, and complex or large search spaces. • Due to its powerful global search capability and low demand for domain expertise, EC has found widespread application in image segmentation. • Classified into five main categories, including region-based, classification-based, clustering-based, thresholding-based, and other types. Graph-based, edge- detection-based and model-based image segmentation techniques are included in this category.
  11. 11. IMAGE SEGMENTATION Image Source - https://ieeexplore.ieee.org/document/9943992
  12. 12. IMAGE SEGMENTATION A. Threshold-based Image Segmentation Methods  most common and straightforward methods for segmenting images.  The basic notion is to determine an image's histogram or other statistical value, then divide the image by comparing it to the histogram or statistical value and grouping pixels that are similar. Multiple threshold values are frequently required to achieve good segmentation performance for complex images.  Achieved by - Optimize threshold values & Optimize other parameters
  13. 13. IMAGE SEGMENTATION B. Region-based Image Segmentation Methods  By preserving the spatial relationships between pixels, region-based image segmentation techniques typically locate regions with similar pixel values that comply with pre-determined rules. Region-based strategies can be broken down into two categories: region-growing strategies and region-merging strategies. A bottom-up method known as a region-growing method uses some pixels as seed pixels and grows the region by checking pixels that are nearby. The neighboring pixels will be placed in the same region as the seed pixel if they comply with predetermined rules. Instead, the region merging technique is a top-down approach that divides an image into smaller sections and then merges such sections in accordance with established guidelines. EC techniques is used to optimize both region-growing and region-merging methods.
  14. 14. IMAGE SEGMENTATION C. Clustering-based Image Segmentation Methods  To achieve image segmentation, clustering-based techniques focuses to combine pixels into distinct clusters with low inter-cluster and high intra-cluster similarity.  EC strategies have been utilized to track down great beginning bunch centroids for picture division.  The number of clusters and the initial cluster centroids are evolved using EC methods.  Optimizing clustering-based image segmentation techniques is a popular application of EC techniques, such as GAs, PSO, DE, and EMO. The number of clusters, initial cluster centers, or both can be maximized using these strategies. In these strategies, a genuine worth vector-based portrayal is ordinarily used to address the arrangements. ***genetic algorithms (GAs);differential evolution (DE) ;particle swarm optimization (PSO);ant colony optimization (ACO) ;genetic programming (GP) ;evolutionary multi-objective optimization (EMO).
  15. 15. IMAGE SEGMENTATION D. Classification-based Image Segmentation Methods  For image segmentation, classification-based methods typically create models or classifiers that can associate each pixel in an image with distinct classes.  The two main types of EC-based image segmentation techniques are those that use EC (mostly GP) to automatically evolve/construct classifiers/models or those that use EC to improve existing classification algorithms. For image segmentation, GP is the primary method that does not rely on any other segmentation or learning techniques to automatically evolve models or classifiers. superior to the other four approaches. Classification-based approaches typically make use of EC-based approaches to optimize CNN-based models and use GP to automatically evolve models and classifiers. Most of the time, GP-based methods find tree-based models with functions and terminals or features that group pixels into different classes and are easy to understand. The goal of the EC-based CNN approaches is to locate promising CNNs for effective image segmentation. The limitations of the existing CNN methods are addressed by these methods.
  16. 16. IMAGE FEATURE ANALYSIS • Numerous tasks involving images rely on image feature analysis. Roasters of raw pixels, which may contain information that is useful or meaningful, are typically used to represent images. • The primary focus of image feature analysis is the examination of images' meaningful features and information. Typically, it consists of feature selection, feature extraction, feature construction, or a more general step known as feature learning. Image Source - https://ieeexplore.ieee.org/document/9943992
  17. 17. IMAGE FEATURE ANALYSIS • A necessary step in image analysis is feature selection, which selects the most significant features from a set of features that have already been extracted. When there are a lot of features and complicated interactions between them, feature selection becomes difficult.  GA-based Methods  PSO-based Methods  ACO-based Methods  EMO-based Methods  Hybrid Methods  Other Methods
  18. 18. IMAGE FEATURE ANALYSIS • Feature Extraction and Learning - ex interactions between features. Optimization of Existing Feature Extraction Methods Automatic Evolution of Models/Descriptors for Feature Extraction
  19. 19. IMAGE CLASSIFICATION In CV and machine learning, image classification is a central and crucial task. Based on the content of the image, it aims to classify it into one of several predetermined categories. A. EC for Evolving NNs for Image Classification: DNNs, or deep CNNs, for image classification have recently been developed using EC methods with success.  Evolving NN Weights and/or Architectures:  Suganuma and co propose the CGP-CNN approach, which uses ResNet blocks or standard convolutional operators to create a tree-based representation of NAS.  Low model complexity and high classification performance are achieved by this approach.  Li introduces a binary encoding to evolve deep CNNs using quantum-behaving PSO.  Gong et al. suggests to combine backpropagation with a co-evolutionary approach to search the parameters of DNN models for image classification.
  20. 20. IMAGE CLASSIFICATION When one method is unable to optimize the training objective function, the weights are learned using co-evolution and backpropagation. For efficient search, the parameter learning task in co-evolution is broken down into numerous smaller tasks. The conventional DNN parameter learning methods are inferior to this one.  EMO for Evolving NNs: In federated learning, Zhu and Jin propound an EMO-based strategy for optimizing NN structures.
  21. 21. IMAGE CLASSIFICATION Multilayer perceptron and CNN models can be optimized using this approach at lower communication costs. Under the real-time federated learning framework, Zhu and Jin develop an EMO-based NAS method with the objectives of minimizing local payload and maximizing model performance. Using double-sampling, this method can effectively cut costs associated with computation and communication. Wang and co. present a PSO-based multi-objective NAS strategy that simultaneously maximizes two goals, namely the number of floating-point operations and classification accuracy.
  22. 22. IMAGE CLASSIFICATION  Computation Efficiency in EC for Evolving NN: Because it requires numerous fitness evaluations, utilizing EC to optimize NNs is computationally costly. As a result, latest research focuses on developing search efficiency-enhancing evaluation strategies that are computationally inexpensive. Zhang et al. propose a node inheritance strategy and a small sample of training data in an evolutionary NAS method to lower the cost of fitness evaluation computations.  Lu, et al. propose NSGANetV2, which makes use of surrogates to predict how CNN models will perform over time. To naturally select one of four surrogate models for fitness prediction, a method of selection known as adaptive switching is proposed.
  23. 23. IMAGE CLASSIFICATION B. EC for Evolving Non-NN-based Methods: Evolving classifiers from Image Features:  For multi-class object classification using domain-independent features, Zhang and Smart propose a GP method with a fitness function that calculates the overlap of the distributions of all possible pairs of classes.  Multiple GP classifiers and the normal probability density function serve as the foundation for the classification decision. Three datasets show good results for this method. Evolving Classifiers from Raw Pixels:  To classify images, the most common approach employs GP to create tree-like models from raw pixels. In GP, binary classifiers based on raw pixels have been developed using multi-tier or multi-layer tree structures.  Al-Sahaf and other present a two-tier GP method with a classification and an aggregation tier: The image classifier is the first, which focuses on identifying small image regions and extracting features.
  24. 24. IMAGE CLASSIFICATION  On four datasets, this method achieves high accuracy, and the learned classifiers and models are easy to understand. Feature Learning and Emerging Topics:  With only a few training images, GP-based methods have demonstrated tremendous potential for image classification.  Al Sahaf and other propose a compound GP for region detection and a one-shot GP for developing classifiers directly from images to make feature extraction for binary image classification easier.  Only a few training images are needed for this method, which performs better than traditional methods.  Bi and co. In order to enhance GP's generalizability for few-shot image classification, we propose a dual-tree-based GP approach and develop an effective fitness function.  On nine image datasets, this method outperforms popular few-shot learning techniques in the 3-shot and 5-shot scenarios.
  25. 25. OBJECT DETECTION • In CV and image analysis, object detection is a fundamental function. It's not just about finding objects, but also where they are in the images. • Object detection has been made possible by using EC methods with success. Using EC methods to naturally derive detectors from scratch and using EC methods to optimize an object detection system are two broad categories of existing methods. A. Optimizing Object Detection Systems –  EC methods can be used to improve the features, parameters, and positions of objects in an object detection system, among other things.
  26. 26. OBJECT DETECTION  An entropic binary PSO method for enhancing an ear detection system's entropy map. Based on the optimized entropy map, classification is carried out using a threshold. On four face image datasets, the performance of this method is promising.  Eye tracking and detection based on PSO. The deformable multiple template matching algorithm's center point and scaling factor are optimized using PSO in this approach.  Notable item identification is an exceptional instance of item recognition that just recognizes the main item in a picture and overlooks other superfluous ones. Singh and co. propose a PSO approach to obtaining a saliency map for object detection and the best weights for three features. The PSO search is guided by an evaluation of the background and attention pixels by a fitness function. Different performance metrics show that this method performs better than other methods.  Moghaddam and co propose a GP-based technique to develop undeniable level highlights from low - level saliency highlights for remarkable item recognition. A GP method is used to construct features from each of the four subsets of the feature sets, which are then combined to produce the final saliency map.  GP-based methods combine the saliency feature maps in non-linear ways, in contrast to PSO-based methods.
  27. 27. OBJECT DETECTION B. Automatically Evolving Detectors – Automatically evolving image object detectors have also utilized EC techniques. Based on GP, the primary methods for evolving detectors. Zhang investigates three feature sets, creates a novel fitness function, and develops a two-phase learning strategy to propose a novel GP-based object detection method. Using a small portion of the training samples, GP-based detectors are developed and evaluated in the first phase with the goal of increasing classification accuracy. Using all of the training samples, the evolved detectors are further refined in the second phase with the goal of increasing detection accuracy. For object detection, the findings demonstrate that the two-stage GP method is more reliable and efficient than the single-stage GP method. Object detection is typically more challenging than these other tasks because it requires object localization and classification. However, EC techniques have demonstrated a lot of promise for both automatically evolving object detectors and improving existing object detection systems.
  28. 28. OTHER TASKS • Other image-related tasks, such as interest point detection, image registration, remote-sensing image classification, and object tracking, have demonstrated significant potential for EC methods.  Interest Point Detection –  The detection of interesting points, such as corners, blobs, and edges, that convey relevant or interesting visual information is referred to as interest points detection.  GP is used to develop interest point detectors in images automatically. The majority of the works are by Gustavo Olague's group. In order to construct detectors from images, Trujillo and Olague devise a GP method that incorporates arithmetic functions and image operators like histogram normalization and Gaussian derivatives.  Point dispersion, stability,and information content are few of the many possible competing goals in interest point detection.  For the purpose of interest point detection, a multi-objective GP method with the goals of stability and point dispersion optimization is investigated.
  29. 29. OTHER TASKS Image Registration –  In many applications, like medical imaging, image registration is an important preprocessing step that aims to align two or more images by finding the best transformation of the images in geometric space.  Image registration, specifically locating the best transformation, has been made possible by the development of EC methods.  In order to simultaneously look for control parameters and image registration solution, a self- adaptive evolutionary optimization (SaEvO) algorithm is presented.  because the EC-based applications to image registration have not received a lot of attention in recent years, even though EC techniques have powerful search capabilities and flexible representations.
  30. 30. OTHER TASKS Remote Sensing Image Classification – Typically, the goal of remote sensing or hyperspectral image classification is to classify individual pixels within images rather than the entire image. In this context, numerous EC-based methods have also been developed to carry out tasks like parameter optimization, feature subset or band selection, and locating the best cluster centers for the clustering-based methods. The interval type-2 semi-supervised possibilistic fuzzy C-means clustering method for satellite image classification employs a PSO-based method to optimize the cluster centers and parameters. Most of the methods compared to this one achieve better accuracy. The ACO method employs a binary encoding to select a subset of the 76 original features, including the Gabor wavelet, the grey-level co-occurrence matrix (GLCM), and histogram statistics, while simultaneously locating the best SVM parameters. The performance of this approach is superior to that of other EC-based approaches. For cropland field classification on very high-resolution images, use GP to develop classifiers based on spectral, shape, and texture features. On two datasets, this approach outperforms five conventional classifiers.
  31. 31. OTHER TASKS Object Tracking–  The goal of object tracking is to locate the positions of objects in a series of images or frames in order to follow them. Typically, a target object is presented, and the objective is to locate it across multiple frames.  Half and half gravitational pursuit calculation (HGSA) to look for the areas of items by expanding a cosine-closeness capability that assesses the comparability between the objective items and likely articles in the picture utilizing highlights extricated by CNNs. A cutting-edge SI-based object tracker becomes more accurate as a result of this approach.  GP method for developing detectors that can track moving targets against a background that is stationary. This GP strategy involves pixels in a sliding window as terminals and a few administrators/capabilities to develop the finders/classifiers. Under various conditions, such as noise, the proposed method seems to be working.  an EC-based NAS method for one-time object tracking that uses a pretrained supernet to find the best trackers. Compared to the earlier methods that did not include NAS search, this one is more effective and efficient.
  32. 32. APPLICATIONS Facial image analysis Biomedical image analysis Remote sensing image analysis Image analysis related to humans Agricultural image analysis Others. e.g., motor imagery classification , art classification , digit recognition , kinship verification , fish classification, and vehicle detection
  33. 33. CHALLENGES Scalability Representation Search Mechanisms Interpretability Computational Cost Recognition by the Main CV Community and Publishing Papers in Major CV and AI Conferences
  34. 34. TRENDS Evolutionary Deep Learning (EDL). Computationally Efficient ECV Methods. EMO for CV and Image Analysis. EC with Transfer Learning for CV and Image Analysis. ECV Using Small-Scale Training Data. EC Methods for Interpretable CV and Image Analysis.
  35. 35. CONCLUSIONS Even though EC-based methods frequently perform well, they still have to deal with a number of problems and limitations regarding scalability, representations, search mechanisms, computational cost, and interpretability. For CNNs, EC-based approaches may be a good option because they don't require a lot of expertise in the NN domain, require expensive GPU computing resources, and have poor interpretability. It is evident that EC-based approaches have demonstrated great potential when applied to a wide range of image-related tasks.
  36. 36. REFERENCES • A Survey on Evolutionary Computation for Computer Vision and Image Analysis: Past, Present, and Future Trends; Ying Bi, Bing Xue, Pablo Mesejo, Stefano Cagnoni, Mengjie Zhang.https://arxiv.org/abs/2209.06399v1 • Learning Adaptive Evolutionary Computation for Solving Multi-Objective Optimization Problems. Remco Coppens, Robbert Reijnen, Yingqian Zhang, Laurens Bliek, Berend Steenhuisen, https://arxiv.org/abs/2211.09719
  37. 37. THANK YOU

Editor's Notes

  • It is still a very rapidly developing area, where many new research and techniques are being proposed to effectively solve different tasks.
  • The above chart shows the taxonomy and tasks that are in scope for our survey. In our survey, we will just consider ECs categorized by TASK TYPE.
  • Scalability: Scalability is a common issue in most EC-based methods for image analysis. In recent years, big data have become a trend in image analysis, where the number of images in the datasets is very large (see the datasets summarised in the supplementary materials). Some well-known image/object classification datasets include the CIFAR10, CIFAR100 and ImageNet datasets.
    Representation: A careful design of representations is essential to the success of EC-based image analysis. Several different representations are used in EC methods, including string-based, vector-based, tree-based, and graph-based, allowing EC methods to solve a variety of image-analysis tasks. Even within the same specific task, EC methods may use multiple representations. Since representations are typically task-dependent, their design is challenging.
    Search Mechanisms: The underlying search mechanisms are at the heart of EC methods. A good search mechanism can better balance exploration and exploitation and find globally optimal solutions. Image analysis tasks are typically very difficult, requiring powerful search mechanisms.
    Interpretability: Interpretability is very important in many image analysis tasks, such as biomedical image analysis. However, traditional methods are affected by poorer performance, domain knowledge requirements, and poor flexibility. A trade-off can be found by making the use of domain knowledge more flexible and automatic by using EC methods.
    Computational Cost: Computational cost is an essential factor for EC-based image-analysis applications. Compared with random or exhaustive search methods, EC methods are by far less computationally expensive. In some cases, their powerful search ability makes them even faster than traditional CV methods. However, in many supervised learning tasks, such as image classification, EC methods may be computationally expensive. Some attempts to improve the computational efficiency of EC-based image analysis have used surrogates to predict fitness values.
    Recognition by the Main CV Community and Publishing Papers in Major CV and AI Conferences: Despite the above technical challenges, other important challenges include the awareness of the contributions of EC-based methods by the CV community and publishing EC-based works in major CV and AI conferences. For the EC-based contributions to be fully recognised by the major CV community, it is necessary to publish at these major conferences. However, this is very challenging due to the low acceptance rates.

×