The document proposes a pixel-level fusion approach using vision transformer for early detection of Alzheimer's disease from MRI and PET scans. It applies discrete wavelet transform to decompose MRI and PET images into frequency bands. It then uses a pre-trained VGG16 model to optimize the decomposition. The frequency bands are fused using inverse discrete wavelet transform. Finally, a pre-trained vision transformer model classifies the fused images and achieves 81.25% accuracy for AD/EMCI and AD/LMCI classification on MRI data and 93.75% accuracy on PET data, outperforming existing methods on PET data classification.
An effective feature selection using improved marine predators algorithm for ...IJECEIAES
Alzheimer’s disease (AD) is an irremediable neurodegenerative illness developed by the fast deterioration of brain cells. AD is mostly common in elder people and it extremely disturbs the physical and mental health of patients, therefore early detection is essential to prevent AD development. However, the precise detection of AD and mild cognitive impairment (MCI) is difficult during classification. In this paper, the Residual network i.e., ResNet-18 is used for extracting the features, and the proposed improved marine predators algorithm (IMPA) is developed for choosing the optimum features to perform an effective classification of AD. The multi-verse optimizer (MVO) used in the IMPA helps to balance exploration and exploitation, which leads to the selection of optimal relevant features. Further, the classification of AD is accomplished using the multiclass support vector machine (MSVM). Open access series of imaging studies-1 (OASIS-1) and Alzheimer disease neuroimaging initiative (ADNI) datasets are used to evaluate the IMPA-MSVM method. The performance of the IMPA-MSVM method is analyzed using accuracy, sensitivity, specificity, positive predictive value (PPV) and matthews correlation coefficient (MCC). The existing methods such as the deep learning-based segmenting method using SegNet (DLSS), mish activation function (MAF) with spatial transformer network (STN) and BrainNet2D are used to evaluate the IMPA-MSVM method. The accuracy of IMPA-MSVM for the ADNI dataset is 98.43% which is more when compared to the DLSS and MAF-STN.
Alzheimer’s detection through neuro imaging and subsequent fusion for clinica...IJECEIAES
This document summarizes a study that aims to detect Alzheimer's disease through neuroimaging and subsequently fuse magnetic resonance imaging (MRI) and positron emission tomography (PET) scans. The study first detects Alzheimer's from MRI scans by segmenting gray and white matter and calculating their volume ratios. Image fusion is then performed on the MRI scan detected with Alzheimer's and a corresponding PET scan. The fusion combines the two scans into a single image containing structural and anatomical information to aid clinical diagnosis. The proposed method achieved good performance metrics for the fused image, with a peak signal-to-noise ratio of 60.6 dB, mean square error of 0.0176, and other favorable values, suggesting it yields more informative results than individual scans
Alzheimer Disease Prediction using Machine Learning AlgorithmsIRJET Journal
This document summarizes research on using machine learning algorithms to predict Alzheimer's disease. It discusses collecting data from the Alzheimer's Disease Neuroimaging Initiative database and preprocessing MRI images. Feature extraction is performed using techniques like volume and thickness measurements. Machine learning models like CNNs and SVMs are trained on the data and tested to distinguish between patient groups and predict progression. The research aims to more accurately diagnose Alzheimer's at earlier stages by combining clinical assessments with structural neuroimaging data and machine learning. Accuracy of over 90% was achieved in some cases at distinguishing between patient classifications like AD vs normal control.
The IoT and registration of MRI brain diagnosis based on genetic algorithm an...IJEECSIAES
The technology of the multimodal brain image registration is the key method for accurate and rapid diagnosis and treatment of brain diseases. For achieving high-resolution image registration, a fast sub pixel registration algorithm is used based on single-step discrete wavelet transform (DWT) combined with phase convolution neural network (CNN) to classify the registration of brain tumors. In this work apply the genetic algorithm and CNN clasifcation in registration of magnetic resonance imaging (MRI) image. This approach follows eight steps, reading the source of MRI brain image and loading the reference image, enhencment all MRI images by bilateral filter, transforming DWT image by applying the DWT2, evaluating (fitness function) each MRI image by using entropy, applying the genetic algorithm, by selecting the two images based on rollout wheel and crossover of the two images, the CNN classify the result of subtraction to normal or abnormal, “in the eighth one,” the Arduino and global system for mobile (GSM) 8080 are applied to send the message to patient. The proposed model is tested on MRI Medical City Hospital in Baghdad database consist 550 normal and 350 abnormal and split to 80% training and 20 testing, the proposed model result achieves the 98.8% accuracy.
The IoT and registration of MRI brain diagnosis based on genetic algorithm an...nooriasukmaningtyas
The document describes a proposed model for MRI brain diagnosis using genetic algorithms, convolutional neural networks, and the Internet of Things. The model has eight steps: loading MRI images, enhancing images, applying discrete wavelet transform, evaluating images using entropy, applying genetic algorithm for registration, subtracting images and using CNN to classify results as normal or abnormal, and sending messages to patients using Arduino and GSM. The model was tested on 550 normal and 350 abnormal MRI images, achieving 98.8% accuracy in classification.
A deep learning approach for brain tumor detection using magnetic resonance ...IJECEIAES
The growth of abnormal cells in the brain’s tissue causes brain tumors. Brain tumors are considered one of the most dangerous disorders in children and adults. It develops quickly, and the patient’s survival prospects are slim if not appropriately treated. Proper treatment planning and precise diagnoses are essential to improving a patient’s life expectancy. Brain tumors are mainly diagnosed using magnetic resonance imaging (MRI). As part of a convolution neural network (CNN)-based illustration, an architecture containing five convolution layers, five max-pooling layers, a Flatten layer, and two dense layers has been proposed for detecting brain tumors from MRI images. The proposed model includes an automatic feature extractor, modified hidden layer architecture, and activation function. Several test cases were performed, and the proposed model achieved 98.6% accuracy and 97.8% precision score with a low cross-entropy rate. Compared with other approaches such as adjacent feature propagation network (AFPNet), mask region-based CNN (mask RCNN), YOLOv5, and Fourier CNN (FCNN), the proposed model has performed better in detecting brain tumors.
The biomedical profession has gained importance due to the rapid and accurate diagnosis of clinical patients using computer-aided diagnosis (CAD) tools.
The diagnosis and treatment of Alzheimer’s disease (AD) using complementary multimodalities can improve the quality of life and mental state of patients.
In this study, we integrated a lightweight custom convolutional neural network
(CNN) model and nature-inspired optimization techniques to enhance the performance, robustness, and stability of progress detection in AD. A multi-modal
fusion database approach was implemented, including positron emission tomography (PET) and magnetic resonance imaging (MRI) datasets, to create a fused
database. We compared the performance of custom and pre-trained deep learning models with and without optimization and found that employing natureinspired algorithms like the particle swarm optimization algorithm (PSO) algorithm significantly improved system performance. The proposed methodology,
which includes a fused multimodality database and optimization strategy, improved performance metrics such as training, validation, test accuracy, precision, and recall. Furthermore, PSO was found to improve the performance of
pre-trained models by 3-5% and custom models by up to 22%. Combining different medical imaging modalities improved the overall model performance by
2-5%. In conclusion, a customized lightweight CNN model and nature-inspired
optimization techniques can significantly enhance progress detection, leading to
better biomedical research and patient care.
Hippocampus’s volume calculation on coronal slice’s for strengthening the dia...TELKOMNIKA JOURNAL
Alzheimer’s is one of the most common types of dementia in the world. Although not a contagious disease, this disease has many impacts, especially in socio-economic life. In diagnosing Alzheimer’s and using interview techniques, physical examination methods are also used, namely using an magnetic resonance imaging (MRI) machine to get a clear image of the patient’s brain condition, with a focus on the hippocampus and ventricular area. In this paper, we discuss the calculation of the volume of the hippocampus, especially the coronal slice, to provide information to doctors in making decisions on diagnosing the severity of Alzheimer’s. Using the basis of volume calculations, we made a 3D visualization reconstruction of the coronal hippocampus slice area in order to make it easier for doctors to analyze the condition of the hippocampus area, which in the end will be used as a recommendation in the classification of the severity of Alzheimer’s. Our experimental results show, the lower the severity, the bigger the volume, the more slices, and the longer the counting time.
An effective feature selection using improved marine predators algorithm for ...IJECEIAES
Alzheimer’s disease (AD) is an irremediable neurodegenerative illness developed by the fast deterioration of brain cells. AD is mostly common in elder people and it extremely disturbs the physical and mental health of patients, therefore early detection is essential to prevent AD development. However, the precise detection of AD and mild cognitive impairment (MCI) is difficult during classification. In this paper, the Residual network i.e., ResNet-18 is used for extracting the features, and the proposed improved marine predators algorithm (IMPA) is developed for choosing the optimum features to perform an effective classification of AD. The multi-verse optimizer (MVO) used in the IMPA helps to balance exploration and exploitation, which leads to the selection of optimal relevant features. Further, the classification of AD is accomplished using the multiclass support vector machine (MSVM). Open access series of imaging studies-1 (OASIS-1) and Alzheimer disease neuroimaging initiative (ADNI) datasets are used to evaluate the IMPA-MSVM method. The performance of the IMPA-MSVM method is analyzed using accuracy, sensitivity, specificity, positive predictive value (PPV) and matthews correlation coefficient (MCC). The existing methods such as the deep learning-based segmenting method using SegNet (DLSS), mish activation function (MAF) with spatial transformer network (STN) and BrainNet2D are used to evaluate the IMPA-MSVM method. The accuracy of IMPA-MSVM for the ADNI dataset is 98.43% which is more when compared to the DLSS and MAF-STN.
Alzheimer’s detection through neuro imaging and subsequent fusion for clinica...IJECEIAES
This document summarizes a study that aims to detect Alzheimer's disease through neuroimaging and subsequently fuse magnetic resonance imaging (MRI) and positron emission tomography (PET) scans. The study first detects Alzheimer's from MRI scans by segmenting gray and white matter and calculating their volume ratios. Image fusion is then performed on the MRI scan detected with Alzheimer's and a corresponding PET scan. The fusion combines the two scans into a single image containing structural and anatomical information to aid clinical diagnosis. The proposed method achieved good performance metrics for the fused image, with a peak signal-to-noise ratio of 60.6 dB, mean square error of 0.0176, and other favorable values, suggesting it yields more informative results than individual scans
Alzheimer Disease Prediction using Machine Learning AlgorithmsIRJET Journal
This document summarizes research on using machine learning algorithms to predict Alzheimer's disease. It discusses collecting data from the Alzheimer's Disease Neuroimaging Initiative database and preprocessing MRI images. Feature extraction is performed using techniques like volume and thickness measurements. Machine learning models like CNNs and SVMs are trained on the data and tested to distinguish between patient groups and predict progression. The research aims to more accurately diagnose Alzheimer's at earlier stages by combining clinical assessments with structural neuroimaging data and machine learning. Accuracy of over 90% was achieved in some cases at distinguishing between patient classifications like AD vs normal control.
The IoT and registration of MRI brain diagnosis based on genetic algorithm an...IJEECSIAES
The technology of the multimodal brain image registration is the key method for accurate and rapid diagnosis and treatment of brain diseases. For achieving high-resolution image registration, a fast sub pixel registration algorithm is used based on single-step discrete wavelet transform (DWT) combined with phase convolution neural network (CNN) to classify the registration of brain tumors. In this work apply the genetic algorithm and CNN clasifcation in registration of magnetic resonance imaging (MRI) image. This approach follows eight steps, reading the source of MRI brain image and loading the reference image, enhencment all MRI images by bilateral filter, transforming DWT image by applying the DWT2, evaluating (fitness function) each MRI image by using entropy, applying the genetic algorithm, by selecting the two images based on rollout wheel and crossover of the two images, the CNN classify the result of subtraction to normal or abnormal, “in the eighth one,” the Arduino and global system for mobile (GSM) 8080 are applied to send the message to patient. The proposed model is tested on MRI Medical City Hospital in Baghdad database consist 550 normal and 350 abnormal and split to 80% training and 20 testing, the proposed model result achieves the 98.8% accuracy.
The IoT and registration of MRI brain diagnosis based on genetic algorithm an...nooriasukmaningtyas
The document describes a proposed model for MRI brain diagnosis using genetic algorithms, convolutional neural networks, and the Internet of Things. The model has eight steps: loading MRI images, enhancing images, applying discrete wavelet transform, evaluating images using entropy, applying genetic algorithm for registration, subtracting images and using CNN to classify results as normal or abnormal, and sending messages to patients using Arduino and GSM. The model was tested on 550 normal and 350 abnormal MRI images, achieving 98.8% accuracy in classification.
A deep learning approach for brain tumor detection using magnetic resonance ...IJECEIAES
The growth of abnormal cells in the brain’s tissue causes brain tumors. Brain tumors are considered one of the most dangerous disorders in children and adults. It develops quickly, and the patient’s survival prospects are slim if not appropriately treated. Proper treatment planning and precise diagnoses are essential to improving a patient’s life expectancy. Brain tumors are mainly diagnosed using magnetic resonance imaging (MRI). As part of a convolution neural network (CNN)-based illustration, an architecture containing five convolution layers, five max-pooling layers, a Flatten layer, and two dense layers has been proposed for detecting brain tumors from MRI images. The proposed model includes an automatic feature extractor, modified hidden layer architecture, and activation function. Several test cases were performed, and the proposed model achieved 98.6% accuracy and 97.8% precision score with a low cross-entropy rate. Compared with other approaches such as adjacent feature propagation network (AFPNet), mask region-based CNN (mask RCNN), YOLOv5, and Fourier CNN (FCNN), the proposed model has performed better in detecting brain tumors.
The biomedical profession has gained importance due to the rapid and accurate diagnosis of clinical patients using computer-aided diagnosis (CAD) tools.
The diagnosis and treatment of Alzheimer’s disease (AD) using complementary multimodalities can improve the quality of life and mental state of patients.
In this study, we integrated a lightweight custom convolutional neural network
(CNN) model and nature-inspired optimization techniques to enhance the performance, robustness, and stability of progress detection in AD. A multi-modal
fusion database approach was implemented, including positron emission tomography (PET) and magnetic resonance imaging (MRI) datasets, to create a fused
database. We compared the performance of custom and pre-trained deep learning models with and without optimization and found that employing natureinspired algorithms like the particle swarm optimization algorithm (PSO) algorithm significantly improved system performance. The proposed methodology,
which includes a fused multimodality database and optimization strategy, improved performance metrics such as training, validation, test accuracy, precision, and recall. Furthermore, PSO was found to improve the performance of
pre-trained models by 3-5% and custom models by up to 22%. Combining different medical imaging modalities improved the overall model performance by
2-5%. In conclusion, a customized lightweight CNN model and nature-inspired
optimization techniques can significantly enhance progress detection, leading to
better biomedical research and patient care.
Hippocampus’s volume calculation on coronal slice’s for strengthening the dia...TELKOMNIKA JOURNAL
Alzheimer’s is one of the most common types of dementia in the world. Although not a contagious disease, this disease has many impacts, especially in socio-economic life. In diagnosing Alzheimer’s and using interview techniques, physical examination methods are also used, namely using an magnetic resonance imaging (MRI) machine to get a clear image of the patient’s brain condition, with a focus on the hippocampus and ventricular area. In this paper, we discuss the calculation of the volume of the hippocampus, especially the coronal slice, to provide information to doctors in making decisions on diagnosing the severity of Alzheimer’s. Using the basis of volume calculations, we made a 3D visualization reconstruction of the coronal hippocampus slice area in order to make it easier for doctors to analyze the condition of the hippocampus area, which in the end will be used as a recommendation in the classification of the severity of Alzheimer’s. Our experimental results show, the lower the severity, the bigger the volume, the more slices, and the longer the counting time.
A modified residual network for detection and classification of Alzheimer’s ...IJECEIAES
Alzheimer's disease (AD) is a brain disease that significantly declines a person's ability to remember and behave normally. By applying several approaches to distinguish between various stages of AD, neuroimaging data has been used to extract different patterns associated with various phases of AD. However, because the brain patterns of older adults and those in different phases are similar, researchers have had difficulty classifying them. In this paper, the 50-layer residual neural network (ResNet) is modified by adding extra convolution layers to make the extracted features more diverse. Besides, the activation function (ReLU) was replaced with (Leaky ReLU) because ReLU takes the negative parts of its input, drops them to zero, and retains the positive parts. These negative inputs may contain useful feature information that could aid in the development of high-level discriminative features. Thus, Leaky ReLU was used instead of ReLU to prevent any potential loss of input information. In order to train the network from scratch without encountering the issue of overfitting, we added a dropout layer before the fully connected layer. The proposed method successfully classified the four stages of AD with an accuracy of 97.49 % and 98 % for precision, recall, and f1-score.
Glioblastomas brain tumour segmentation based on convolutional neural network...IJECEIAES
Brain tumour segmentation can improve diagnostics efficiency, rise the prediction rate and treatment planning. This will help the doctors and experts in their work. Where many types of brain tumour may be classified easily, the gliomas tumour is challenging to be segmented because of the diffusion between the tumour and the surrounding edema. Another important challenge with this type of brain tumour is that the tumour may grow anywhere in the brain with different shape and size. Brain cancer presents one of the most famous diseases over the world, which encourage the researchers to find a high-throughput system for tumour detection and classification. Several approaches have been proposed to design automatic detection and classification systems. This paper presents an integrated framework to segment the gliomas brain tumour automatically using pixel clustering for the MRI images foreground and background and classify its type based on deep learning mechanism, which is the convolutional neural network. In this work, a novel segmentation and classification system is proposed to detect the tumour cells and classify the brain image if it is healthy or not. After collecting data for healthy and non-healthy brain images, satisfactory results are found and registered using computer vision approaches. This approach can be used as a part of a bigger diagnosis system for breast tumour detection and manipulation.
A deep learning approach based on stochastic gradient descent and least absol...nooriasukmaningtyas
More than eighty-five to ninety percentage of the diabetic patients are affected with diabetic retinopathy (DR) which is an eye disorder that leads to blindness. The computational techniques can support to detect the DR by using the retinal images. However, it is hard to measure the DR with the raw retinal image. This paper proposes an effective method for identification of DR from the retinal images. In this research work, initially the Weiner filter is used for preprocessing the raw retinal image. Then the preprocessed image is segmented using fuzzy c-mean technique. Then from the segmented image, the features are extracted using grey level co-occurrence matrix (GLCM). After extracting the fundus image, the feature selection is performed stochastic gradient descent, and least absolute shrinkage and selection operator (LASSO) for accurate identification during the classification process. Then the inception v3-convolutional neural network (IV3-CNN) model is used in the classification process to classify the image as DR image or non-DR image. By applying the proposed method, the classification performance of IV3-CNN model in identifying DR is studied. Using the proposed method, the DR is identified with the accuracy of about 95%, and the processed retinal image is identified as mild DR.
Evaluation of Default Mode Network In Mild Cognitive Impairment and Alzheimer...CSCJournals
Although progressive functional brain network disorders has been one of the indication of Alzheimer's disease, The current research on aging and dementia focus on diagnostics of the cognitive changes of normal aging and Alzheimer Disease (AD), these changes known as Mild Cognitive Impairment (MCI). The default mode network (DMN) is a network of interacting brain regions known to have activity highly correlated with each other and distinct from other networks in the brain, the default mode network is active during passive rest and consists of a set of brain areas that are tightly functionally connected and distinct from other systems within the brain. Anatomically, the DMN includes the posterior cingulated cortex (PCC), dorsal and ventral medial prefrontal cortex, the lateral parietal cortex, and the medial temporal lobes. DMN involves multiple anatomical networks that converge on cortical hubs, such as the PCC, ventral medial prefrontal, and inferior parietal cortices. The aim of this study was to evaluate the default mode network functional connectivity in MCI patients. While no treatments are recommended for MCI currently, Mild Cognitive Impairment is becoming a very important subject for researchers and deserves more recognition and further study, In order to increase the ability to recognize earlier symptoms of Alzheimer's disease.
Hybrid model for detection of brain tumor using convolution neural networksCSITiaesprime
The development of aberrant brain cells, some of which may turn cancerous, is known as a brain tumor. Magnetic resonance imaging (MRI) scans are the most common technique for finding brain tumors. Information about the aberrant tissue growth in the brain is discernible from the MRI scans. In numerous research papers, machine learning, and deep learning algorithms are used to detect brain tumors. It takes extremely little time to forecast a brain tumor when these algorithms are applied to MRI pictures, and better accuracy makes it easier to treat patients. The radiologist can make speedy decisions because of this forecast. The proposed work creates a hybrid convolution neural networks (CNN) model using CNN for feature extraction and logistic regression (LR). The pre-trained model visual geometry group 16 (VGG16) is used for the extraction of features. To reduce the complexity and parameters to train we eliminated the last eight layers of VGG16. From this transformed model the features are extracted in the form of a vector array. These features fed into different machine learning classifiers like support vector machine (SVM), naïve bayes (NB), LR, extreme gradient boosting (XGBoost), AdaBoost, and random forest for training and testing. The performance of different classifiers is compared. The CNN-LR hybrid combination outperformed the remaining classifiers. The evaluation measures such as recall, precision, F1-score, and accuracy of the proposed CNN-LR model are 94%, 94%, 94%, and 91% respectively.
Primary challenges are the identification, segmentation, and extraction of the afflicted area from the scanning of magnetic resonance. However, it is a time-consuming and tiresome for clinical specialists. In this paper,
an automated brain tumor system is proposed. The proposed system employs hybrid image processing techniques such as contrast correction, histogram normalization, thresholding techniques, arithmetic, and morphological operations to quarantine nearby organs and other tissue from the brain for improving the localization of the affected region. At first, the skull stripping process is proposed to segregate the non-designated regions to extract the designated brain regions. Those resultant brain region images are further subjected to discover the brain tumor. The planned scheme is studied on the magnetic resonance (MR) images with the use of T1, T2, T1c, and fluid-attenuated inversion recovery (FLAIR). The proposed hybrid method employed. The results reveal that the proposed method is quite efficient to extract the tumor region. The accuracy rate for segmentation and separation of area of interest in brain tumor reached to 95%. Finally, the significance of the proposed procedure is confirmed using the real image clinical dataset got from ten patients were diagnosed as begin, malignant, and metastatic brain tumors in Al-Yarmouk and Baghdad teaching hospital in Baghdad, Iraq.
A Review On Methods For Feature Extraction And Classification For The Automat...Heather Strinden
This document reviews various feature extraction and classification methods that have been used for the automated detection of Alzheimer's disease from magnetic resonance imaging (MRI) scans. It summarizes several studies that used different feature extraction techniques like voxel-based, vertex-based, and region of interest-based methods. Popular classification algorithms discussed include support vector machines, linear discriminant analysis, Bayesian classifiers and artificial neural networks. The document concludes that selecting relevant features extracted from MRI scans can yield accurate classification of Alzheimer's disease.
Ensemble learning approach for multi-class classification of Alzheimer’s stag...TELKOMNIKA JOURNAL
The document presents an ensemble learning approach for multi-class classification of Alzheimer's disease stages using magnetic resonance imaging (MRI). The proposed algorithm concatenates the output layers of three pre-trained models - Xception, InceptionV3, and MobileNet. The algorithm is tested on MRI images from the Alzheimer's Disease Neuroimaging Initiative database. The middle slice is extracted from each 3D MRI and data augmentation is applied. The ensemble model provides 85% accuracy for multi-class classification of Alzheimer's, mild cognitive impairment convertible, mild cognitive impairment non-convertible, and cognitively normal. It also achieves 85% accuracy for classifying mild cognitive impairment convertible vs non-convertible, 94% for Alzheimer's vs
Automatic brain tumor detection using adaptive region growing with thresholdi...IAESIJAI
Brain cancer affects many people around the world. It's not just limited to the elderly; it is also recognized in children. With the development of image processing, early detection of mental development is possible. Some designers suggest deformable models, histogram averaging, or manual division. Due to constant manual intervention, these cycles can be uncomfortable and tiring. This research introduces a high-level system for the removal of malignant tumors from attractive reverberation images, based on a programmed and rapid distribution strategy for surface extraction and recreation for clinicians. To test the proposed system, acquired tomography images from the Cancer Imaging Archive were used. The results of the study strongly demonstrate that the intended structure is viable in brain tumor detection.
Comparison of resting electroencephalogram coherence in patients with mild co...IJECEIAES
Mild cognitive impairment (MCI) was a condition beginning before more serious deterioration, leading to Alzheimer’s dementia (AD). MCI detection was needed to determine the patient's therapeutic management. Analysis of electroencephalogram (EEG) coherence is one of the modalities for MCI detection. Therefore, this study investigated the inter and intra-hemispheric coherence over 16 EEG channels in the frequency range of 1-30 Hz. The simulation results showed that most of the electrode pair coherence in MCI patients have decreased compared to normal elderly subjects. In inter hemisphere coherence, significant differences (p<0.05) were found in the FP1-FP2 electrode pairs. Meanwhile, significant differences (p<0.05) were found in almost all pre-frontal area connectivity of the intra-hemisphere coherence pairs. The electrode pairs were FP2-F4, FP2-T4, FP1-F3, FP1-F7, FP1-C3, FP1-T3, FP1-P3, FP1-T5, FP1-O1, F3-O1, and T3-T5. The decreased coherence in MCI patients showed the disconnection of cortical connections as a result of the death of the neurons. Furthermore, the coherence value can be used as a multimodal feature in normal elderly subjects and MCI. It is hoped that current studies may be considered for early detection of Alzheimer’s in a larger population.Mild cognitive impairment (MCI) was a condition beginning before more serious deterioration, leading to Alzheimer’s dementia (AD). MCI detection was needed to determine the patient's therapeutic management. Analysis of electroencephalogram (EEG) coherence is one of the modalities for MCI detection. Therefore, this study investigated the inter and intra-hemispheric coherence over 16 EEG channels in the frequency range of 1-30 Hz. The simulation results showed that most of the electrode pair coherence in MCI patients have decreased compared to normal elderly subjects. In inter hemisphere coherence, significant differences (p<0.05) were found in the FP1-FP2 electrode pairs. Meanwhile, significant differences (p<0.05) were found in almost all pre-frontal area connectivity of the intra-hemisphere coherence pairs. The electrode pairs were FP2-F4, FP2-T4, FP1-F3, FP1-F7, FP1-C3, FP1-T3, FP1-P3, FP1-T5, FP1-O1, F3-O1, and T3-T5. The decreased coherence in MCI patients showed the disconnection of cortical connections as a result of the death of the neurons. Furthermore, the coherence value can be used as a multimodal feature in normal elderly subjects and MCI. It is hoped that current studies may be considered for early detection of Alzheimer’s in a larger population.
Brain Image Fusion using DWT and Laplacian Pyramid Approach and Tumor Detecti...INFOGAIN PUBLICATION
Image fusion is the process of combining important information from two or more images into a single image. The resulting image will be more enhanced than any of the input pictures. The idea of combining multiple image modalities to furnish a single, more enhanced image is well established, special fusion methods have been proposed in literature. This paper is based on image fusion using laplacian pyramid and Discreet Wavelet Transform (DWT) methods. This system uses an easy and effective algorithm for multi-focus image fusion which uses fusion rules to create fused image. Subsequently, the fused image is obtained by applying inverse discreet wavelet transform. After fused image is obtained, watershed segmentation algorithm is applied to detect the tumor part in fused image.
Early Detection of Alzheimer’s Disease Using Machine Learning TechniquesIRJET Journal
This document discusses early detection of Alzheimer's disease using machine learning techniques. It proposes using deep learning models to classify brain MRI images from three planes (coronal, axial, sagittal) to detect brain damage related to Alzheimer's with 99.5% accuracy. This high-performing model is compared to other state-of-the-art machine and deep learning models. The methodology involves training deep learning and machine learning models on MRI datasets and evaluating their performance on test data.
This document summarizes a study that used deep learning techniques to develop an automatic system for assessing Alzheimer's disease diagnosis based on sagittal magnetic resonance images (MRIs). The study used sagittal MRIs from two datasets and conducted experiments with transfer learning to achieve accurate results. The study showed that Alzheimer's disease damages and stages can be distinguished in sagittal MRIs, and results were similar to state-of-the-art methods using horizontal MRIs. This suggests sagittal MRIs could be effectively used to identify early-stage Alzheimer's disease. Transfer learning was also shown to be essential for developing models with limited training data.
Early Stage Detection of Alzheimer’s Disease Using Deep LearningIRJET Journal
This document presents a study on early detection of Alzheimer's disease using deep learning techniques. The authors build an end-to-end framework for classifying MRI images into four stages of Alzheimer's - mild demented, very mild demented, moderate demented, and non-demented. They use transfer learning with pre-trained models like VGG16 and ResNet50, achieving accuracies of 95% and 84% respectively. A custom CNN model is also developed, achieving 93% accuracy. The models are trained on a dataset of 6400 MRI images from Kaggle. A web application is also created for remote analysis and detection of Alzheimer's stage to help doctors and patients.
An ensemble deep learning classifier of entropy convolutional neural network ...BASMAJUMAASALEHALMOH
This document summarizes a research paper that proposes an ensemble deep learning classifier for efficient disease prediction of heart disease and breast cancer. The classifier combines two models: an Entropy Convolutional Neural Network (ECNN) and a Divergence Weight Bidirectional LSTM (DWBi-LSTM). Feature selection is first performed on the dataset using Bayesian Network imputation and Mode Fuzzy Weight Canonical Polyadic decomposition. An Adaptive Mean Chaotic Grey Wolf Optimization algorithm is then used for wrapper-based feature selection. The results of the ECNN and DWBi-LSTM classifiers are combined using bootstrap aggregation and majority voting. Performance is evaluated using various metrics and compared to existing classifiers. The proposed ensemble classifier aims to improve disease prediction accuracy over single classifiers.
This document reviews various automated techniques that have been developed for brain tumor detection. It summarizes research done by several researchers on methods like sequential floating forward selection, color coding schemes using brain atlases, neural networks, region growing segmentation combined with area calculation, symmetry analysis of tumor areas in MRI images, and combining clustering and classification algorithms. The paper concludes that image segmentation plays an important role in medical applications like tumor diagnosis and that more robust techniques are needed for high accuracy and reliability.
A Survey On Identification Of Multiple Sclerosis Lesions From Brain MRIJanelle Martinez
This document summarizes several papers on using MRI and image analysis techniques to identify multiple sclerosis (MS) lesions in the brain. It discusses using metrics like fractional anisotropy and apparent diffusion coefficient from diffusion tensor imaging to analyze changes in normal-appearing white matter. It also reviews methods like texture analysis using gray-level co-occurrence matrices, convolutional neural networks, and phase and orientation analysis to segment and identify MS lesions in MRI scans. The goal is to characterize subtle microscopic changes in nerve fibers caused by MS through analysis of MRI textures and alignments to improve diagnosis and understanding of the disease.
MR Image Segmentation of Patients’ Brain Using Disease Specific a priori Know...CSCJournals
Segmentation of high quality brain MR images using a priori knowledge about brain structures enables a more accurate and comprehensive interpretation. Benefits of applying a priori knowledge about the brain structures may also be employed for image segmentation of specific brain and neural patients. Such procedure may be performed to determine the disease stage or monitor its gradual progression over time. However segmenting brain images of patients using general a priori knowledge which corresponds to healthy subjects would result in inaccurate and unreliable interpretation in the regions which are affected by the disease. In this paper, a technique is proposed for extracting a priori knowledge about structural distribution of different brain tissues affected by a specific disease to be applied for accurate segmentation of the patients’ brain images. For this purpose, extracted a priori knowledge is gradually represented as disease specific probability maps throughout an iterative process, and then is utilized in a statistical approach for segmentation of new patients’ images. Experiments conducted on a large set of images acquired from patients with a similar neurodegenerative disease implied success of the proposed technique for representing meaningful a priori knowledge as disease specific probability maps. Promising results obtained also indicated an accurate segmentation of brain MR images of the new patients using the represented a priori knowledge, into three tissue classes of gray matter, white matter, and cerebrospinal fluid. This enables an accurate estimation of tissues’ thickness and volumes and can be counted as a substantial forward step for more reliable monitoring and interpretation of progression in specific brain and neural diseases.
SEGMENTATION OF MULTIPLE SCLEROSIS LESION IN BRAIN MR IMAGES USING FUZZY C-MEANSijaia
Magnetic resonance images (MRI) play an important role in supporting and substituting clinical
information in the diagnosis of multiple sclerosis (MS) disease by presenting lesion in brain MR images. In
this paper, an algorithm for MS lesion segmentation from Brain MR Images has been presented. We revisit
the modification of properties of fuzzy -c means algorithms and the canny edge detection. By changing and
reformed fuzzy c-means clustering algorithms, and applying canny contraction principle, a relationship
between MS lesions and edge detection is established. For the special case of FCM, we derive a sufficient
condition and clustering parameters, allowing identification of them as (local) minima of the objective
function.
Survey of various methods used for integrating machine learning into brain tu...Drjabez
This document surveys various machine learning methods used for integrating machine learning into brain tumor detection and classification from MRI images. It discusses preprocessing techniques like median filtering, Gaussian high pass filtering, and morphology dilation to enhance images. Segmentation techniques covered include thresholding, edge detection, region-based, watershed, Berkeley wavelet transform, K-means clustering, and neural networks. Feature extraction calculates correlation, skewness. Classification algorithms discussed are multi-layer perceptron, naive Bayes, and support vector machines. The document provides an overview of key steps and methods for machine learning-based brain tumor detection and segmentation from MRI images.
International Conference on NLP, Artificial Intelligence, Machine Learning an...gerogepatton
International Conference on NLP, Artificial Intelligence, Machine Learning and Applications (NLAIM 2024) offers a premier global platform for exchanging insights and findings in the theory, methodology, and applications of NLP, Artificial Intelligence, Machine Learning, and their applications. The conference seeks substantial contributions across all key domains of NLP, Artificial Intelligence, Machine Learning, and their practical applications, aiming to foster both theoretical advancements and real-world implementations. With a focus on facilitating collaboration between researchers and practitioners from academia and industry, the conference serves as a nexus for sharing the latest developments in the field.
A modified residual network for detection and classification of Alzheimer’s ...IJECEIAES
Alzheimer's disease (AD) is a brain disease that significantly declines a person's ability to remember and behave normally. By applying several approaches to distinguish between various stages of AD, neuroimaging data has been used to extract different patterns associated with various phases of AD. However, because the brain patterns of older adults and those in different phases are similar, researchers have had difficulty classifying them. In this paper, the 50-layer residual neural network (ResNet) is modified by adding extra convolution layers to make the extracted features more diverse. Besides, the activation function (ReLU) was replaced with (Leaky ReLU) because ReLU takes the negative parts of its input, drops them to zero, and retains the positive parts. These negative inputs may contain useful feature information that could aid in the development of high-level discriminative features. Thus, Leaky ReLU was used instead of ReLU to prevent any potential loss of input information. In order to train the network from scratch without encountering the issue of overfitting, we added a dropout layer before the fully connected layer. The proposed method successfully classified the four stages of AD with an accuracy of 97.49 % and 98 % for precision, recall, and f1-score.
Glioblastomas brain tumour segmentation based on convolutional neural network...IJECEIAES
Brain tumour segmentation can improve diagnostics efficiency, rise the prediction rate and treatment planning. This will help the doctors and experts in their work. Where many types of brain tumour may be classified easily, the gliomas tumour is challenging to be segmented because of the diffusion between the tumour and the surrounding edema. Another important challenge with this type of brain tumour is that the tumour may grow anywhere in the brain with different shape and size. Brain cancer presents one of the most famous diseases over the world, which encourage the researchers to find a high-throughput system for tumour detection and classification. Several approaches have been proposed to design automatic detection and classification systems. This paper presents an integrated framework to segment the gliomas brain tumour automatically using pixel clustering for the MRI images foreground and background and classify its type based on deep learning mechanism, which is the convolutional neural network. In this work, a novel segmentation and classification system is proposed to detect the tumour cells and classify the brain image if it is healthy or not. After collecting data for healthy and non-healthy brain images, satisfactory results are found and registered using computer vision approaches. This approach can be used as a part of a bigger diagnosis system for breast tumour detection and manipulation.
A deep learning approach based on stochastic gradient descent and least absol...nooriasukmaningtyas
More than eighty-five to ninety percentage of the diabetic patients are affected with diabetic retinopathy (DR) which is an eye disorder that leads to blindness. The computational techniques can support to detect the DR by using the retinal images. However, it is hard to measure the DR with the raw retinal image. This paper proposes an effective method for identification of DR from the retinal images. In this research work, initially the Weiner filter is used for preprocessing the raw retinal image. Then the preprocessed image is segmented using fuzzy c-mean technique. Then from the segmented image, the features are extracted using grey level co-occurrence matrix (GLCM). After extracting the fundus image, the feature selection is performed stochastic gradient descent, and least absolute shrinkage and selection operator (LASSO) for accurate identification during the classification process. Then the inception v3-convolutional neural network (IV3-CNN) model is used in the classification process to classify the image as DR image or non-DR image. By applying the proposed method, the classification performance of IV3-CNN model in identifying DR is studied. Using the proposed method, the DR is identified with the accuracy of about 95%, and the processed retinal image is identified as mild DR.
Evaluation of Default Mode Network In Mild Cognitive Impairment and Alzheimer...CSCJournals
Although progressive functional brain network disorders has been one of the indication of Alzheimer's disease, The current research on aging and dementia focus on diagnostics of the cognitive changes of normal aging and Alzheimer Disease (AD), these changes known as Mild Cognitive Impairment (MCI). The default mode network (DMN) is a network of interacting brain regions known to have activity highly correlated with each other and distinct from other networks in the brain, the default mode network is active during passive rest and consists of a set of brain areas that are tightly functionally connected and distinct from other systems within the brain. Anatomically, the DMN includes the posterior cingulated cortex (PCC), dorsal and ventral medial prefrontal cortex, the lateral parietal cortex, and the medial temporal lobes. DMN involves multiple anatomical networks that converge on cortical hubs, such as the PCC, ventral medial prefrontal, and inferior parietal cortices. The aim of this study was to evaluate the default mode network functional connectivity in MCI patients. While no treatments are recommended for MCI currently, Mild Cognitive Impairment is becoming a very important subject for researchers and deserves more recognition and further study, In order to increase the ability to recognize earlier symptoms of Alzheimer's disease.
Hybrid model for detection of brain tumor using convolution neural networksCSITiaesprime
The development of aberrant brain cells, some of which may turn cancerous, is known as a brain tumor. Magnetic resonance imaging (MRI) scans are the most common technique for finding brain tumors. Information about the aberrant tissue growth in the brain is discernible from the MRI scans. In numerous research papers, machine learning, and deep learning algorithms are used to detect brain tumors. It takes extremely little time to forecast a brain tumor when these algorithms are applied to MRI pictures, and better accuracy makes it easier to treat patients. The radiologist can make speedy decisions because of this forecast. The proposed work creates a hybrid convolution neural networks (CNN) model using CNN for feature extraction and logistic regression (LR). The pre-trained model visual geometry group 16 (VGG16) is used for the extraction of features. To reduce the complexity and parameters to train we eliminated the last eight layers of VGG16. From this transformed model the features are extracted in the form of a vector array. These features fed into different machine learning classifiers like support vector machine (SVM), naïve bayes (NB), LR, extreme gradient boosting (XGBoost), AdaBoost, and random forest for training and testing. The performance of different classifiers is compared. The CNN-LR hybrid combination outperformed the remaining classifiers. The evaluation measures such as recall, precision, F1-score, and accuracy of the proposed CNN-LR model are 94%, 94%, 94%, and 91% respectively.
Primary challenges are the identification, segmentation, and extraction of the afflicted area from the scanning of magnetic resonance. However, it is a time-consuming and tiresome for clinical specialists. In this paper,
an automated brain tumor system is proposed. The proposed system employs hybrid image processing techniques such as contrast correction, histogram normalization, thresholding techniques, arithmetic, and morphological operations to quarantine nearby organs and other tissue from the brain for improving the localization of the affected region. At first, the skull stripping process is proposed to segregate the non-designated regions to extract the designated brain regions. Those resultant brain region images are further subjected to discover the brain tumor. The planned scheme is studied on the magnetic resonance (MR) images with the use of T1, T2, T1c, and fluid-attenuated inversion recovery (FLAIR). The proposed hybrid method employed. The results reveal that the proposed method is quite efficient to extract the tumor region. The accuracy rate for segmentation and separation of area of interest in brain tumor reached to 95%. Finally, the significance of the proposed procedure is confirmed using the real image clinical dataset got from ten patients were diagnosed as begin, malignant, and metastatic brain tumors in Al-Yarmouk and Baghdad teaching hospital in Baghdad, Iraq.
A Review On Methods For Feature Extraction And Classification For The Automat...Heather Strinden
This document reviews various feature extraction and classification methods that have been used for the automated detection of Alzheimer's disease from magnetic resonance imaging (MRI) scans. It summarizes several studies that used different feature extraction techniques like voxel-based, vertex-based, and region of interest-based methods. Popular classification algorithms discussed include support vector machines, linear discriminant analysis, Bayesian classifiers and artificial neural networks. The document concludes that selecting relevant features extracted from MRI scans can yield accurate classification of Alzheimer's disease.
Ensemble learning approach for multi-class classification of Alzheimer’s stag...TELKOMNIKA JOURNAL
The document presents an ensemble learning approach for multi-class classification of Alzheimer's disease stages using magnetic resonance imaging (MRI). The proposed algorithm concatenates the output layers of three pre-trained models - Xception, InceptionV3, and MobileNet. The algorithm is tested on MRI images from the Alzheimer's Disease Neuroimaging Initiative database. The middle slice is extracted from each 3D MRI and data augmentation is applied. The ensemble model provides 85% accuracy for multi-class classification of Alzheimer's, mild cognitive impairment convertible, mild cognitive impairment non-convertible, and cognitively normal. It also achieves 85% accuracy for classifying mild cognitive impairment convertible vs non-convertible, 94% for Alzheimer's vs
Automatic brain tumor detection using adaptive region growing with thresholdi...IAESIJAI
Brain cancer affects many people around the world. It's not just limited to the elderly; it is also recognized in children. With the development of image processing, early detection of mental development is possible. Some designers suggest deformable models, histogram averaging, or manual division. Due to constant manual intervention, these cycles can be uncomfortable and tiring. This research introduces a high-level system for the removal of malignant tumors from attractive reverberation images, based on a programmed and rapid distribution strategy for surface extraction and recreation for clinicians. To test the proposed system, acquired tomography images from the Cancer Imaging Archive were used. The results of the study strongly demonstrate that the intended structure is viable in brain tumor detection.
Comparison of resting electroencephalogram coherence in patients with mild co...IJECEIAES
Mild cognitive impairment (MCI) was a condition beginning before more serious deterioration, leading to Alzheimer’s dementia (AD). MCI detection was needed to determine the patient's therapeutic management. Analysis of electroencephalogram (EEG) coherence is one of the modalities for MCI detection. Therefore, this study investigated the inter and intra-hemispheric coherence over 16 EEG channels in the frequency range of 1-30 Hz. The simulation results showed that most of the electrode pair coherence in MCI patients have decreased compared to normal elderly subjects. In inter hemisphere coherence, significant differences (p<0.05) were found in the FP1-FP2 electrode pairs. Meanwhile, significant differences (p<0.05) were found in almost all pre-frontal area connectivity of the intra-hemisphere coherence pairs. The electrode pairs were FP2-F4, FP2-T4, FP1-F3, FP1-F7, FP1-C3, FP1-T3, FP1-P3, FP1-T5, FP1-O1, F3-O1, and T3-T5. The decreased coherence in MCI patients showed the disconnection of cortical connections as a result of the death of the neurons. Furthermore, the coherence value can be used as a multimodal feature in normal elderly subjects and MCI. It is hoped that current studies may be considered for early detection of Alzheimer’s in a larger population.Mild cognitive impairment (MCI) was a condition beginning before more serious deterioration, leading to Alzheimer’s dementia (AD). MCI detection was needed to determine the patient's therapeutic management. Analysis of electroencephalogram (EEG) coherence is one of the modalities for MCI detection. Therefore, this study investigated the inter and intra-hemispheric coherence over 16 EEG channels in the frequency range of 1-30 Hz. The simulation results showed that most of the electrode pair coherence in MCI patients have decreased compared to normal elderly subjects. In inter hemisphere coherence, significant differences (p<0.05) were found in the FP1-FP2 electrode pairs. Meanwhile, significant differences (p<0.05) were found in almost all pre-frontal area connectivity of the intra-hemisphere coherence pairs. The electrode pairs were FP2-F4, FP2-T4, FP1-F3, FP1-F7, FP1-C3, FP1-T3, FP1-P3, FP1-T5, FP1-O1, F3-O1, and T3-T5. The decreased coherence in MCI patients showed the disconnection of cortical connections as a result of the death of the neurons. Furthermore, the coherence value can be used as a multimodal feature in normal elderly subjects and MCI. It is hoped that current studies may be considered for early detection of Alzheimer’s in a larger population.
Brain Image Fusion using DWT and Laplacian Pyramid Approach and Tumor Detecti...INFOGAIN PUBLICATION
Image fusion is the process of combining important information from two or more images into a single image. The resulting image will be more enhanced than any of the input pictures. The idea of combining multiple image modalities to furnish a single, more enhanced image is well established, special fusion methods have been proposed in literature. This paper is based on image fusion using laplacian pyramid and Discreet Wavelet Transform (DWT) methods. This system uses an easy and effective algorithm for multi-focus image fusion which uses fusion rules to create fused image. Subsequently, the fused image is obtained by applying inverse discreet wavelet transform. After fused image is obtained, watershed segmentation algorithm is applied to detect the tumor part in fused image.
Early Detection of Alzheimer’s Disease Using Machine Learning TechniquesIRJET Journal
This document discusses early detection of Alzheimer's disease using machine learning techniques. It proposes using deep learning models to classify brain MRI images from three planes (coronal, axial, sagittal) to detect brain damage related to Alzheimer's with 99.5% accuracy. This high-performing model is compared to other state-of-the-art machine and deep learning models. The methodology involves training deep learning and machine learning models on MRI datasets and evaluating their performance on test data.
This document summarizes a study that used deep learning techniques to develop an automatic system for assessing Alzheimer's disease diagnosis based on sagittal magnetic resonance images (MRIs). The study used sagittal MRIs from two datasets and conducted experiments with transfer learning to achieve accurate results. The study showed that Alzheimer's disease damages and stages can be distinguished in sagittal MRIs, and results were similar to state-of-the-art methods using horizontal MRIs. This suggests sagittal MRIs could be effectively used to identify early-stage Alzheimer's disease. Transfer learning was also shown to be essential for developing models with limited training data.
Early Stage Detection of Alzheimer’s Disease Using Deep LearningIRJET Journal
This document presents a study on early detection of Alzheimer's disease using deep learning techniques. The authors build an end-to-end framework for classifying MRI images into four stages of Alzheimer's - mild demented, very mild demented, moderate demented, and non-demented. They use transfer learning with pre-trained models like VGG16 and ResNet50, achieving accuracies of 95% and 84% respectively. A custom CNN model is also developed, achieving 93% accuracy. The models are trained on a dataset of 6400 MRI images from Kaggle. A web application is also created for remote analysis and detection of Alzheimer's stage to help doctors and patients.
An ensemble deep learning classifier of entropy convolutional neural network ...BASMAJUMAASALEHALMOH
This document summarizes a research paper that proposes an ensemble deep learning classifier for efficient disease prediction of heart disease and breast cancer. The classifier combines two models: an Entropy Convolutional Neural Network (ECNN) and a Divergence Weight Bidirectional LSTM (DWBi-LSTM). Feature selection is first performed on the dataset using Bayesian Network imputation and Mode Fuzzy Weight Canonical Polyadic decomposition. An Adaptive Mean Chaotic Grey Wolf Optimization algorithm is then used for wrapper-based feature selection. The results of the ECNN and DWBi-LSTM classifiers are combined using bootstrap aggregation and majority voting. Performance is evaluated using various metrics and compared to existing classifiers. The proposed ensemble classifier aims to improve disease prediction accuracy over single classifiers.
This document reviews various automated techniques that have been developed for brain tumor detection. It summarizes research done by several researchers on methods like sequential floating forward selection, color coding schemes using brain atlases, neural networks, region growing segmentation combined with area calculation, symmetry analysis of tumor areas in MRI images, and combining clustering and classification algorithms. The paper concludes that image segmentation plays an important role in medical applications like tumor diagnosis and that more robust techniques are needed for high accuracy and reliability.
A Survey On Identification Of Multiple Sclerosis Lesions From Brain MRIJanelle Martinez
This document summarizes several papers on using MRI and image analysis techniques to identify multiple sclerosis (MS) lesions in the brain. It discusses using metrics like fractional anisotropy and apparent diffusion coefficient from diffusion tensor imaging to analyze changes in normal-appearing white matter. It also reviews methods like texture analysis using gray-level co-occurrence matrices, convolutional neural networks, and phase and orientation analysis to segment and identify MS lesions in MRI scans. The goal is to characterize subtle microscopic changes in nerve fibers caused by MS through analysis of MRI textures and alignments to improve diagnosis and understanding of the disease.
MR Image Segmentation of Patients’ Brain Using Disease Specific a priori Know...CSCJournals
Segmentation of high quality brain MR images using a priori knowledge about brain structures enables a more accurate and comprehensive interpretation. Benefits of applying a priori knowledge about the brain structures may also be employed for image segmentation of specific brain and neural patients. Such procedure may be performed to determine the disease stage or monitor its gradual progression over time. However segmenting brain images of patients using general a priori knowledge which corresponds to healthy subjects would result in inaccurate and unreliable interpretation in the regions which are affected by the disease. In this paper, a technique is proposed for extracting a priori knowledge about structural distribution of different brain tissues affected by a specific disease to be applied for accurate segmentation of the patients’ brain images. For this purpose, extracted a priori knowledge is gradually represented as disease specific probability maps throughout an iterative process, and then is utilized in a statistical approach for segmentation of new patients’ images. Experiments conducted on a large set of images acquired from patients with a similar neurodegenerative disease implied success of the proposed technique for representing meaningful a priori knowledge as disease specific probability maps. Promising results obtained also indicated an accurate segmentation of brain MR images of the new patients using the represented a priori knowledge, into three tissue classes of gray matter, white matter, and cerebrospinal fluid. This enables an accurate estimation of tissues’ thickness and volumes and can be counted as a substantial forward step for more reliable monitoring and interpretation of progression in specific brain and neural diseases.
SEGMENTATION OF MULTIPLE SCLEROSIS LESION IN BRAIN MR IMAGES USING FUZZY C-MEANSijaia
Magnetic resonance images (MRI) play an important role in supporting and substituting clinical
information in the diagnosis of multiple sclerosis (MS) disease by presenting lesion in brain MR images. In
this paper, an algorithm for MS lesion segmentation from Brain MR Images has been presented. We revisit
the modification of properties of fuzzy -c means algorithms and the canny edge detection. By changing and
reformed fuzzy c-means clustering algorithms, and applying canny contraction principle, a relationship
between MS lesions and edge detection is established. For the special case of FCM, we derive a sufficient
condition and clustering parameters, allowing identification of them as (local) minima of the objective
function.
Survey of various methods used for integrating machine learning into brain tu...Drjabez
This document surveys various machine learning methods used for integrating machine learning into brain tumor detection and classification from MRI images. It discusses preprocessing techniques like median filtering, Gaussian high pass filtering, and morphology dilation to enhance images. Segmentation techniques covered include thresholding, edge detection, region-based, watershed, Berkeley wavelet transform, K-means clustering, and neural networks. Feature extraction calculates correlation, skewness. Classification algorithms discussed are multi-layer perceptron, naive Bayes, and support vector machines. The document provides an overview of key steps and methods for machine learning-based brain tumor detection and segmentation from MRI images.
International Conference on NLP, Artificial Intelligence, Machine Learning an...gerogepatton
International Conference on NLP, Artificial Intelligence, Machine Learning and Applications (NLAIM 2024) offers a premier global platform for exchanging insights and findings in the theory, methodology, and applications of NLP, Artificial Intelligence, Machine Learning, and their applications. The conference seeks substantial contributions across all key domains of NLP, Artificial Intelligence, Machine Learning, and their practical applications, aiming to foster both theoretical advancements and real-world implementations. With a focus on facilitating collaboration between researchers and practitioners from academia and industry, the conference serves as a nexus for sharing the latest developments in the field.
Introduction- e - waste – definition - sources of e-waste– hazardous substances in e-waste - effects of e-waste on environment and human health- need for e-waste management– e-waste handling rules - waste minimization techniques for managing e-waste – recycling of e-waste - disposal treatment methods of e- waste – mechanism of extraction of precious metal from leaching solution-global Scenario of E-waste – E-waste in India- case studies.
Electric vehicle and photovoltaic advanced roles in enhancing the financial p...IJECEIAES
Climate change's impact on the planet forced the United Nations and governments to promote green energies and electric transportation. The deployments of photovoltaic (PV) and electric vehicle (EV) systems gained stronger momentum due to their numerous advantages over fossil fuel types. The advantages go beyond sustainability to reach financial support and stability. The work in this paper introduces the hybrid system between PV and EV to support industrial and commercial plants. This paper covers the theoretical framework of the proposed hybrid system including the required equation to complete the cost analysis when PV and EV are present. In addition, the proposed design diagram which sets the priorities and requirements of the system is presented. The proposed approach allows setup to advance their power stability, especially during power outages. The presented information supports researchers and plant owners to complete the necessary analysis while promoting the deployment of clean energy. The result of a case study that represents a dairy milk farmer supports the theoretical works and highlights its advanced benefits to existing plants. The short return on investment of the proposed approach supports the paper's novelty approach for the sustainable electrical system. In addition, the proposed system allows for an isolated power setup without the need for a transmission line which enhances the safety of the electrical network
A SYSTEMATIC RISK ASSESSMENT APPROACH FOR SECURING THE SMART IRRIGATION SYSTEMSIJNSA Journal
The smart irrigation system represents an innovative approach to optimize water usage in agricultural and landscaping practices. The integration of cutting-edge technologies, including sensors, actuators, and data analysis, empowers this system to provide accurate monitoring and control of irrigation processes by leveraging real-time environmental conditions. The main objective of a smart irrigation system is to optimize water efficiency, minimize expenses, and foster the adoption of sustainable water management methods. This paper conducts a systematic risk assessment by exploring the key components/assets and their functionalities in the smart irrigation system. The crucial role of sensors in gathering data on soil moisture, weather patterns, and plant well-being is emphasized in this system. These sensors enable intelligent decision-making in irrigation scheduling and water distribution, leading to enhanced water efficiency and sustainable water management practices. Actuators enable automated control of irrigation devices, ensuring precise and targeted water delivery to plants. Additionally, the paper addresses the potential threat and vulnerabilities associated with smart irrigation systems. It discusses limitations of the system, such as power constraints and computational capabilities, and calculates the potential security risks. The paper suggests possible risk treatment methods for effective secure system operation. In conclusion, the paper emphasizes the significant benefits of implementing smart irrigation systems, including improved water conservation, increased crop yield, and reduced environmental impact. Additionally, based on the security analysis conducted, the paper recommends the implementation of countermeasures and security approaches to address vulnerabilities and ensure the integrity and reliability of the system. By incorporating these measures, smart irrigation technology can revolutionize water management practices in agriculture, promoting sustainability, resource efficiency, and safeguarding against potential security threats.
Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapte...University of Maribor
Slides from talk presenting:
Aleš Zamuda: Presentation of IEEE Slovenia CIS (Computational Intelligence Society) Chapter and Networking.
Presentation at IcETRAN 2024 session:
"Inter-Society Networking Panel GRSS/MTT-S/CIS
Panel Session: Promoting Connection and Cooperation"
IEEE Slovenia GRSS
IEEE Serbia and Montenegro MTT-S
IEEE Slovenia CIS
11TH INTERNATIONAL CONFERENCE ON ELECTRICAL, ELECTRONIC AND COMPUTING ENGINEERING
3-6 June 2024, Niš, Serbia
Embedded machine learning-based road conditions and driving behavior monitoringIJECEIAES
Car accident rates have increased in recent years, resulting in losses in human lives, properties, and other financial costs. An embedded machine learning-based system is developed to address this critical issue. The system can monitor road conditions, detect driving patterns, and identify aggressive driving behaviors. The system is based on neural networks trained on a comprehensive dataset of driving events, driving styles, and road conditions. The system effectively detects potential risks and helps mitigate the frequency and impact of accidents. The primary goal is to ensure the safety of drivers and vehicles. Collecting data involved gathering information on three key road events: normal street and normal drive, speed bumps, circular yellow speed bumps, and three aggressive driving actions: sudden start, sudden stop, and sudden entry. The gathered data is processed and analyzed using a machine learning system designed for limited power and memory devices. The developed system resulted in 91.9% accuracy, 93.6% precision, and 92% recall. The achieved inference time on an Arduino Nano 33 BLE Sense with a 32-bit CPU running at 64 MHz is 34 ms and requires 2.6 kB peak RAM and 139.9 kB program flash memory, making it suitable for resource-constrained embedded systems.
Comparative analysis between traditional aquaponics and reconstructed aquapon...bijceesjournal
The aquaponic system of planting is a method that does not require soil usage. It is a method that only needs water, fish, lava rocks (a substitute for soil), and plants. Aquaponic systems are sustainable and environmentally friendly. Its use not only helps to plant in small spaces but also helps reduce artificial chemical use and minimizes excess water use, as aquaponics consumes 90% less water than soil-based gardening. The study applied a descriptive and experimental design to assess and compare conventional and reconstructed aquaponic methods for reproducing tomatoes. The researchers created an observation checklist to determine the significant factors of the study. The study aims to determine the significant difference between traditional aquaponics and reconstructed aquaponics systems propagating tomatoes in terms of height, weight, girth, and number of fruits. The reconstructed aquaponics system’s higher growth yield results in a much more nourished crop than the traditional aquaponics system. It is superior in its number of fruits, height, weight, and girth measurement. Moreover, the reconstructed aquaponics system is proven to eliminate all the hindrances present in the traditional aquaponics system, which are overcrowding of fish, algae growth, pest problems, contaminated water, and dead fish.
Understanding Inductive Bias in Machine LearningSUTEJAS
This presentation explores the concept of inductive bias in machine learning. It explains how algorithms come with built-in assumptions and preferences that guide the learning process. You'll learn about the different types of inductive bias and how they can impact the performance and generalizability of machine learning models.
The presentation also covers the positive and negative aspects of inductive bias, along with strategies for mitigating potential drawbacks. We'll explore examples of how bias manifests in algorithms like neural networks and decision trees.
By understanding inductive bias, you can gain valuable insights into how machine learning models work and make informed decisions when building and deploying them.
Harnessing WebAssembly for Real-time Stateless Streaming PipelinesChristina Lin
Traditionally, dealing with real-time data pipelines has involved significant overhead, even for straightforward tasks like data transformation or masking. However, in this talk, we’ll venture into the dynamic realm of WebAssembly (WASM) and discover how it can revolutionize the creation of stateless streaming pipelines within a Kafka (Redpanda) broker. These pipelines are adept at managing low-latency, high-data-volume scenarios.
TIME DIVISION MULTIPLEXING TECHNIQUE FOR COMMUNICATION SYSTEMHODECEDSIET
Time Division Multiplexing (TDM) is a method of transmitting multiple signals over a single communication channel by dividing the signal into many segments, each having a very short duration of time. These time slots are then allocated to different data streams, allowing multiple signals to share the same transmission medium efficiently. TDM is widely used in telecommunications and data communication systems.
### How TDM Works
1. **Time Slots Allocation**: The core principle of TDM is to assign distinct time slots to each signal. During each time slot, the respective signal is transmitted, and then the process repeats cyclically. For example, if there are four signals to be transmitted, the TDM cycle will divide time into four slots, each assigned to one signal.
2. **Synchronization**: Synchronization is crucial in TDM systems to ensure that the signals are correctly aligned with their respective time slots. Both the transmitter and receiver must be synchronized to avoid any overlap or loss of data. This synchronization is typically maintained by a clock signal that ensures time slots are accurately aligned.
3. **Frame Structure**: TDM data is organized into frames, where each frame consists of a set of time slots. Each frame is repeated at regular intervals, ensuring continuous transmission of data streams. The frame structure helps in managing the data streams and maintaining the synchronization between the transmitter and receiver.
4. **Multiplexer and Demultiplexer**: At the transmitting end, a multiplexer combines multiple input signals into a single composite signal by assigning each signal to a specific time slot. At the receiving end, a demultiplexer separates the composite signal back into individual signals based on their respective time slots.
### Types of TDM
1. **Synchronous TDM**: In synchronous TDM, time slots are pre-assigned to each signal, regardless of whether the signal has data to transmit or not. This can lead to inefficiencies if some time slots remain empty due to the absence of data.
2. **Asynchronous TDM (or Statistical TDM)**: Asynchronous TDM addresses the inefficiencies of synchronous TDM by allocating time slots dynamically based on the presence of data. Time slots are assigned only when there is data to transmit, which optimizes the use of the communication channel.
### Applications of TDM
- **Telecommunications**: TDM is extensively used in telecommunication systems, such as in T1 and E1 lines, where multiple telephone calls are transmitted over a single line by assigning each call to a specific time slot.
- **Digital Audio and Video Broadcasting**: TDM is used in broadcasting systems to transmit multiple audio or video streams over a single channel, ensuring efficient use of bandwidth.
- **Computer Networks**: TDM is used in network protocols and systems to manage the transmission of data from multiple sources over a single network medium.
### Advantages of TDM
- **Efficient Use of Bandwidth**: TDM all
2. Electronics 2023, 12, 1218 2 of 19
magnetic resonance imaging (fMRI) [8–10], fluorodeoxyglucose positron emission tomog-
raphy (FDG-PET) imaging [11,12], and diffusion tensor imaging (DTI) [13] have been used
successfully for an early and accurate diagnosis of AD. To evaluate the progression and
phases of patients with MCI, some existing research has used a single modality successfully.
For the classification between EMCI and cognitive normal (CN), using sMRI, the cortical,
subcortical, and hippocampus subfields were found to be the most discriminating areas of
the brain for AD [14–17]. Unfortunately, the scanned image is occasionally of poor quality
due to focusing errors and noise. As a result, a quality evaluation method is required for
practical use [18].
For fMRI, some features, such as occipital-mid-region, precentral-left, caudate-region,
postcentral-left, and temporal-pole-mid-left are highlighted in the discrimination of EMCI
and NC [19–21]. As for FDG- PET, the average relative cerebral glucose metabolic rate
in the precuneus, superior temporal gyrus in both hemispheres, middle frontal gyrus
and superior frontal gyrus, middle cingulate cortex, and the angular gyrus in the left
hemisphere were examined [22,23]. Because single-modal neuroimaging only offers some
and not all information about brain disorders, it may be insufficient for the classification of
cognitively normal (CN), EMCI, LMCI, and AD people. Deep learning models could be
generalized well and provide increased classification accuracy with the use of multi-modal
neuro-imaging inputs. The alterations in brain structure can be seen in structural MRI
scans of Alzheimer’s patients. PET is a type of functional imaging that can capture the
functioning aspects of the brain to improve the ability to detect lesions. The combination of
MRI and PET is a useful multi-modal neuro-imaging input that can provide more precise
data for clinical diagnosis and treatment.
1.2. Deep Learning Methods for AD Recognition and Classification
Most common models in deep learning that have been tested to have good perfor-
mance in biomedical image segmentation [24,25], medical diagnosis, and image classifi-
cation [26–28] tasks are convolution neural network (CNNs). CNN has greatly been used
in the classification of AD [29,30]. For example, Venugopalan et al. [31] presented multi-
modality intermediate feature level combinations by extracting intermediate features using
a deep learning model, and the resulting features are concatenated and passed through a
classification layer. However, the direct feature concatenation was not trained end-to-end,
thereby not being integrated with the classification step. Sarraf and Tofighi [32]) fused sMRI
and fMRI data to classify AD using LeNet-5 architecture and GoogleNet architecture. Their
method was able to achieve a higher accuracy, but not on MCI stages. Abdelaziz et al. [33]
utilized three stacked CNN models to extract meaningful features from PET, MRI, and
genetic data. Extracted high-level features were sequentially concatenated together for
AD multiclass classification. The proposed method provided missing features for each
incomplete sample using linear interpolation to make the most of the dataset’s available
samples and alleviated data heterogeneity from multimodal data. However, the classifica-
tion accuracy for MCI stages was low. Jin et al. [34] proposed a hybrid three-stage deep
learning framework to classify EMCI and LMCI with incomplete multimodal datasets in
which the classification network was pre-trained via MRI and PET images. The multimodal
data were fused, and the difference between fused data and those of real MRI and PET
data were compared to help the classifier focus on regions for classification for better
classification accuracy.
Although CNN has proven itself well in the above methods, it has always faced
several challenges in AD classification. The CNN model has generalisation and overfitting
problems in binary and multiclass classification [35]. The model can only achieve good
diagnostic results under large and sufficient data, and the model must be deep enough to
extract meaningful information. Forouzannezhad et al. [36] utilized a deep learning model
to detect AD in its early stage using multimodal imaging, combining magnetic resonance
imaging (MRI), positron emission tomography (PET), and standard neuropsychological
test scores with a classification accuracy of 90.3% for binary classification of AD/EMCI.
3. Electronics 2023, 12, 1218 3 of 19
Forouzannezhad et al. [37] further utilized a probabilistic approach to fuse relevant features
from MRI and PET for early detection of AD with 81.4% in EMCI/AD binary classification.
Aqeel et al. [38] used long short-term memory (LSTM) to predict biomarkers (feature
vectors) of AD patients after 6, 12, 21, 18, 24, and 36 months of MRI biomarkers from MRI
and neuropsychological measurements (NM). These anticipated biomarkers are passed
through layers of a fully coupled neural network (NN). The NN layers then determine
whether these biomarkers generated by the LSTM belong to AD patients or MCI patients.
The methodology was tested in the ADNI dataset, and an accuracy of 88.24% was achieved.
Several studies used vision transformer (ViT) for the classification of AD, and it has
offered a lot of advantages in terms of robustness, efficiency, generalization, and transfer
learning compared to CNN. ViTs are more data-efficient than CNNs, which means they
can perform well even in relatively small amounts of data. A key advantage of ViTs is their
ability to perform well on tasks where there is limited data available for training. This is
because ViTs use self-attention mechanisms instead of convolutions, which allows them to
process the entire image at once, rather than in smaller patches. To verify the effectiveness
of ViT on multimodal data of neuroimaging data, several studies attempted and experi-
mentally showed that ViT can achieve the same or even better performance than CNN.
Sarraf et al. [39] utilised an optimised end-to-end pipeline vision transformer to classify AD
stages using rs-fMRI and sMRI data. The proposed model classified AD stages with less
complex architectures with relatively small amounts of data. Fused PET images, consisting
of PET-AV45 and PET-FDG, were trained on ViT for the classification of AD stages [40].
The proposed model achieved a result comparable to CNN models. Kushol et al. [41] took
advantage of the information from both the spatial and frequency domain of MRI images
and proposed a fusion transformer model to fuse the function and structural information
of MRI images. Experimental results showed that the fused information improved the
classification accuracy of AD. Pan and Wang [42] proposed a cross-modal transformer to
achieve a deep fusion of structural and functional information from fMRI and DTI images.
The authors concluded that the proposed model could extract complementary information
from fMRI and DTI, resulting in a higher precision of multimodal connectivity in the
classification of AD than other multimodal fusion techniques [43].
1.3. Contribution, Novelty, and Research Questions
Although the fusion technique in the aforementioned studies is limited to DWT with
ViT, transfer learning can easily be applied to the DWT neuroimaging fusion technique
to provide optimisation that allows improved performance of fused images. To this end,
this study proposed the AD diagnosis method using a pre-trained ViT model to extract
and classify features from fused images from MRI and PET for early detection of AD.
The fused image is achieved by harnessing wavelet decomposition and transfer learning
using VGG16.
The proposed model consists of the following four modules:
1. Discrete wavelet decomposition module to generate one approximate coefficient and
three detail coefficients of MRI and PET image;
2. VGG16 module to generate approximate image and three detail coefficients, which
represent low frequency sub-band and high frequency, respectively;
3. Inverse wavelet transform module to generate fused images on the four bands generated;
4. Pre-trained ViT module that is used to extract and classify features (structural and
functional) from fused image.
To guide our research, we formulate the following research questions:
1. How can MRI and PET data be effectively combined for the early detection of
Alzheimer’s disease?
2. Can a ViT model be trained on the fused data for improved accuracy in the classifica-
tion of Alzheimer’s disease stages (AD/EMCI and AD/LMCI)?
3. How does the proposed multimodal fusion approach compare with existing methods
for the classification of Alzheimer’s disease stages?
4. Electronics 2023, 12, 1218 4 of 19
4. Can the proposed ViT model generalise to new unseen data from the ADNI database
for the classification of Alzheimer’s disease stages?
5. What are the limitations and potential improvements of the proposed multimodal
fusion approach and the use of ViT for medical imaging data analysis?
The novelty of the paper lies in the proposed multimodal fusion approach based on
fine-tuned vision transformer (ViT) for binary classification of AD stages using MRI and
PET data. The approach involves fusing MRI and PET images with adequate preprocessing
and image registration, as well as training a ViT model with the fused data to achieve high
accuracy rates in identifying AD. This model outperforms existing multimodal models in
generalizing well on PET data. The use of a ViT model to analyze medical imaging data is
a novel approach in the field of AD research.
The main contributions of this study include:
• An image fusion technique has been proposed to fuse multimodal images for AD
diagnosis, providing accurate diagnosis of AD to health professionals.
• Complementary information from MRI and PET images is incorporated using wavelet
transform and transfer learning.
• Frequency and location information from MRI and PET images were captured.
• The proposed model is optimised using transfer learning, which improves the perfor-
mance of the proposed model.
The rest of the study is sectioned as follows: The methodology is given in Section 2.
Section 3 gives the experimental analysis, results, and discussion. Section 4 compares
the performance of our proposed model with some CNN models. Finally, we give our
conclusion and recommendation in Section 5.
2. Methods
The proposed architecture for pixel level fusion multimodal vision transformer with
transfer learning is represented in Figure 1.
2.1. Dataset
For experimental analysis, the image datasets of sMRI and FDG-PET images are used,
and this is publicly available in the Alzheimer’s Disease Neuroimaging (ADNI) database.
The sMRI and FDG-PET are both in grayscale and NIFTI format. Participants are selected
from the ADNI2 baseline cohort for our experimental requirement. The age of the subjects
ranges from 60 to 70 years old, including both males and females. Approximately 50 early
EMCI and 50 LMCI participants were selected. T1-weighted volumes and 18F-FDG-PET
images are taken for each participant for the study. The mini-mental state examination
score(MMSE) of 16 and 26 with a Clinical Dement Rating (CDR) of 0.5 was set for selecting
AD, and MMSE of 25 and 28 with a CDR of 0 was set for selecting EMCI participants.
2.2. Preprocessing
Before MRI and PET images can be successfully analyzed, they need to typically
undergo some preprocessing steps to correct for various distortions and artifacts that can
affect the quality of the images. Noise reduction and image registration of images are some
of the preprocessing steps considered in this study. Noise reduction is applied to reduce
the impact of noise on the images. MRI and PET consist of different information, therefore
coregistration of both MRI and PET images is very important for optimal fusion.
2.3. Image Registration
Image registration can be used to align the functional information provided by PET
with structural information provided by MRI. In this study, procrustes analysis [44], which
is a statistical method, is used to align MRI and PET. This is achieved by scaling, rotating,
and translating the images until they are optimally aligned. The procrustes analysis on
MRI and PET is performed by initially indentifying the corresponding points in the two
images. The identified points could be landmarks or features that are present in MRI and
5. Electronics 2023, 12, 1218 5 of 19
PET images. The transformation that aligns the two sets of points is done using singular
value decomposition (SVD). The transformation is applied on a MRI image to align with a
PET image, then the two images are aligned in a common coordinate system. The following
is the mathematical formula for procrustes analysis for MRI and PET utilizing SVD:
X = UXSXV0
X (1)
Y = UYSYV0
Y (2)
where X and Y is the matrix representing MRI data, and PET data respectively, UX and UY
are the singular vectors of the left for the MRI data and PET data, respectively. SX is the
singular value matrix for MRI data, and SY is the singular value matrix for PET data. V0
X
and V0
Y are the right singular vectors for MRI and PET data, respectively.
Figure 1. Block Diagram Integrating Wavelet Decomposition and Transfer Learning for Enhanced
Multimodal Image Fusion for the Proposed Model.
The procrustes analysis is performed by finding the optimal rotation and scaling
factors to align matrix X and matrix Y using equations
R = U0
YUX (3)
6. Electronics 2023, 12, 1218 6 of 19
[P, , Q] = svd(R) (4)
T = PQ0
(5)
where R is the rotation matrix, P and Q are the scaling matrix, and T is the overall transfor-
mation matrix. The transformation matrix is finally applied to the MRI data to align it with
the PET data using Equation (6).
Xaligned = T × X (6)
2.4. Noise Reduction
Noise is introduced into MRI and PET images during the imaging process, resulting in
a blurry appearance of the images. The presence of noise in the images can lead to inaccurate
interpretation of the images, which could lead to incorrect diagnoses or treatment decisions.
Reduction of noise from the images makes the images clearer and more detailed, thereby
allowing for more accurate interpretation. Anisotropic diffusion filtering is used to reduce
the noise fluctuations of the image [45], but it causes blurring of edges, and this could lead
to loss of fine detail and accuracy in the image.
Therefore, we use the wavelet transform that overcomes this challenge by decompos-
ing the image into different frequency bands. Wavelet transform has enabled a new field
that utilizes a DWT equation to eliminate noise from an input image [46]. DWT highlights
important features and eliminates noise. It works by decomposing an image into a series
of small wavelets with localized functions that are used to represent different frequency
components of the image. DWT divides the picture into four subbands (subimages), namely,
LL, LH, LV, and LD, as follows: in its low frequency subband, LL is the approximate image
of the input image; the LH subband extracts the horizontal features of the original image;
the LV subband delivers vertical features; and the LD subband gives diagonal information.
The result of the wavelet decomposition is a series of approximate coefficients for
MRI and PET, known as LL1 and LL2, respectively, as depicted in Figure 1, three detail
coefficients for MRI, known as LH1 (horizontal), LV1 (vertical), and LD1 (diagonal), as
shown in Figure 1, and three detail coefficients for PET, known as LH2 (horizontal), LV2
(vertical), and LD2 (diagonal), as depicted in Figure 1. The horizontal, vertical, and diagonal
detail coefficients represent the image features in different directions. In Figure 1, LL1
and LL2 are the approximation coefficients, which represent the low-frequency, coarse
features of MRI and PET image, respectively, and provide a rough approximation of the
original image. The three detail coefficients represent the high-frequency, fine features,
which provide important information about the image, such as the presence of sharp edges
or sudden changes. Thresholding is used to identify and eliminate coefficients that are due
to noise [47], where coefficients below a certain threshold are set to zero, thereby effectively
removing the noise from the image. Figure 2 showed the low-frequency and the three detail
coefficients for EMCI class from MRI image.
Figure 2. Wavelet transform of the sample EMCI participant MRI image.
7. Electronics 2023, 12, 1218 7 of 19
The mathematical equation DWT(X[n] = ∑ h[k] × X[n − k] + ∑ [k] × X[n − k]) is
replaced by the following :
DWT(f ) = ∑c(a, b)ψ(a, b) × f (x, y) (7)
where DWT(f ) represents the discrete wavelet transform of the image, f (x, y), c(a, b)
represents the coefficients of the wavelet transform, and ψ(a, b) represents the wavelet
basis function.
Equation (1) decomposed the images into various frequency subbands. Using a
thresholding technique, the estimated noise is then eliminated from the subband; values
below a given threshold are regarded as noise and eliminated. The following equation is
used to achieve this:
F0
(x, y) = f (x, y) − λ × ψ(a, b) × c(a, b) (8)
where f 0(x, y) is the denoised image, λ is the threshold value, and ψ(a, b) and c(a, b) are
the same as in Equation (7).
2.5. Multimodal Fusion
The goal of this image fusion is to combine data from PET and MRI scans. Fusion-
based transfer learning is used because it allows for the transfer of knowledge from one
frequency band to another. Transfer learning can learn from multiple sources (different
frequency bands) and adapt to new situations. In this study, the VGG16 network is utilised
to extract features from the four pairs of images from MRI and PET ( LL1 and LL2), (LH1
and LH2), (LV1 and LV2), and ( LD1 and LD2), as shown in Figure 1. For each pair
of features, the VGG16 network is used to interpolate the features and generate a new
band. For instance, for the pair of LL1 and LL2 using the layers of the VGG16 network to
interpolate the features, the resulting output is the LL band. Similarly, for the pair of LH1
and LH2, LV1 and LV2, and LD1 and LD2, using the layers of VGG16 to interpolate their
characteristics, the resulting output is LH, LV, and LD, respectively. LL, LH, LV, and LD
represent the fused features of MRI and PET. To obtain the final fused image, an inverse
discrete wavelet transform ( IDWT) is applied to each of the merged coefficients ( LL, LH,
LV, and LD) and combines the results to form the final fused image. The IDWT uses a set
of wavelet coefficients to reconstruct the final fused image that combines the strength of
both modalities.
Mathematically, the IDWT equation IDWT(c) = IDWT(e1, e2, .. .....en) = ∑n
i=1 ci ∗
φ(2i − 1) is replaced by the following:
f usedImage = IDWT(Merged_Coe f f icients) = IDWT(e1, e2, .., en) =
n
∑
i=1
ei
× φ(2i − 1) (9)
where e1, e2, ..., en are the wavelet coefficients obtained through the DWT. φ(2i − 1) is the
inverse wavelet function corresponding to the level i of the DWT. Merged_Coe f f icients are
the wavelet coefficients obtained by merging the coefficients of the MRI and PET images.
f usedImage is the final fused image obtained through the IDWT.
Figure 3a,b shows original image MRI of EMCI participants and PET of EMCI partici-
pants respectively. Figure 3c shows PET registered image (left) and MRI registered image
(right), while Figure 3d shows the fused image. Eventclick was used to identify the point of
alignments for PET and MRI images. After identifying the points at which the two images
will be aligned, the images are transformed. The approximation and details are shown
in Figure 2. Methods of siaseme are called on LL, LH, LD, and LV images. The trained
network (VGG16) fuses two new input images by passing them through the network and
concatenating their feature representations as the final fused image.
8. Electronics 2023, 12, 1218 8 of 19
(a) (b)
(c) (d)
Figure 3. (a) Original MRI image (EMCI), (b) Original PET image (EMCI), (c) MRI and PET Image
Registration Points for Alignment, (d) Fused Image (EMCI).
2.6. ViT Architecture
The vision transformer architecture consists of a series of self-attention layers, similar
to those found in the transformer architecture. Each self-attention layer processes the input
data in parallel, allowing the network to learn long-range dependencies in the data without
the need for recurrence. The self-attention layers are followed by a series of feed-forward
layers that perform further processing on the output of the self-attention layers. One
of the key innovations of the vision transformer architecture is the use of multi-headed
self-attention, which allows the network to attend to multiple different parts of the input
data simultaneously [48]. This helps the network to learn more complex relationships in
the data and improve its performance on tasks, such as image classification. In addition to
the self-attention layers and feed-forward layers, the vision transformer architecture also
includes several other components, such as a spatial transformer module, which allows the
network to learn spatial relationships in the data, and a learnable pooling module, which
helps to reduce the dimensionality of the input data [49]. We expect that from the fused
images, the ViT models would be able to extract discriminative features related to AD and
thereby make a classification decision. ViT_base_patch16_224 is a medium-sized vision
transformer that processes input images by dividing them into a grid of 16 × 16 patches
used in this study, with each patch being 16 × 16 pixels in size. The model is designed to
take square images with a height and width of 224 pixels as input.
In the proposed approach, we utilized the Timm [50] repository to fine-tune a pre-
trained ViT model. Timm is a library for efficient image classification with PyTorch, which
allows for the fine-tuning of pre-trained models with minimal code. By training a ViT
from scratch, two problems arise: high computational requirements and the need for large
amounts of data. High computational requirements problems occur when there is absence
of access to high performance computing infrastructure. Lack of access to a large dataset
may lead to overfitting and inaccurate results. Fine-tuning a pre-trained ViT model on
a specific task can be used to counter these problems, thereby significantly improving
the performance of the ViT model [51]. Fine-tuned ViT classifies features by using a
9. Electronics 2023, 12, 1218 9 of 19
combination of self-attention and convolutional layers. Fine-tuning a ViT model involves
adjusting the model’s weights and biases to better fit the specific features and characteristics
of the dataset being used. This can be done through a process called back propagation,
which involves adjusting the weights and biases to minimize the error between the model’s
predictions and the true labels of the data [49]. By fine-tuning the model in this way, it can
learn to better classify the features that are relevant to the specific task it is being used for.
The mathematical equation for the back propagation process is as follows:
∆W = α × δ × x (10)
here; ∆W: the update to the weight, α: the learning rate, δ: the error gradient at the output
layer, x: the input to the weight.
The decision function of the pre-trained ViT model on fused image of MRI and PET
images can be expressed mathematically as:
y(i) = ViT(I(i), F, C) (11)
where F = set of fused images, C = Target classes, y(i) = Predicted class for fused image,
ViT = Trained Model, I(i) = Specific fused image in the set F.
3. Experiments and Results
3.1. Experiments
The proposed model implementation and evaluation is performed using pytorch.
The model is trained and validated using a NVIDIA Corporation TU116 (GeForce GTX
1660) graphic processing unit machine. Experiments were performed using five-fold cross-
validation since the fused MRI and PET data are relatively small. The AdamW optimiser
with an initial learning of 0.002, as well as a regularisation technique using weight decay
of 0.01, is utilised in gradient descent. The AdamW optimizer is a modified version of
Adam that integrates weight decay into its update algorithm. By introducing a penalty
term in the loss function that encourages smaller weights, this regularization technique
aids in preventing overfitting. The initial learning rate of 0.002 is a common starting point
for training deep neural networks, as it is typically small enough to prevent the model
from diverging during the early stages of training, but large enough to allow for relatively
rapid convergence to a good solution. Through empirical experimentation and following
best practices, a weight decay value of 0.01 was selected. The batch size is set to 10 when
conducting batch training on NVIDIA GTX 1660 because smaller batch size can lead to
faster convergence and better generalization. A learning rate scheduler is introduced to
adjust the learning rate of the model during training to reduce overfitting. Optimisation of
the learning rate is done using a cyclic learning rate, where the learning rate is not fixed,
but oscillates between maximum and minimum value. The minimum rate is set to 0.002,
and the maximum rate is set to 0.01. The cyclic learning rate is implemented using the
triangular cycle learning rate schedule, where the learning rate increases linearly from a
low value (0.002) to a high value (0.01) over a certain number of iterations (1000) and then
decreases linearly back to the low value over the same number of iterations (1000). We
increase the number of epochs from 10 to 30. Early stopping was further applied to the
model during the training process, and this made the proposed model stop earlier than the
number of epochs (30). At other epochs (10, 20), early stopping did not hold. Increasing the
number of epochs to exceed 30 was shown not be of any advantage since the model did not
complete the 30 epochs because of overfitting.
The fused image from MRI and PET utilized for our proposed multimodal model
combines the high spatial resolution of MRI with the functional information provided by
PET. Original MRI image and original PET image for both classes are compared with the
fused image using structural similarity (SSIM), which incorporate luminance, contrast,
and structure components. The luminance component measures the brightness of the
images, the contrast component measures the difference in intensity between the darkest
10. Electronics 2023, 12, 1218 10 of 19
and lightest pixels, and the structure component measures the similarity of patterns in the
images. To calculate SSIM, we utilized the structural_similarity library, including compo-
nents of the Python library such as skimage. To ensure that the fused image accurately
represents the underlying data in detecting Alzheimer’s disease using fused images of
MRI and PET, it is crucial to consider similarity scores based on luminance, contrast, and
structural components. By selecting similarity scores based on these components, the fused
image can more accurately capture the differences and similarities between the MRI and
PET images. This can make it easier to identify any abnormalities that could indicate
Alzheimer’s disease. The precision of the similarity scores can have a significant impact on
the sensitivity and specificity of AD stages classification.
To assess how well our model performs in classifying the stages of AD using fused
images, we have two sets of experiments: AD vs EMCI, AD vs EMCI, and our set-up. To
evaluate the performance of the proposed multimodal model on the new MRI and PET
datasets of ADNI database participants, accuracy, sensitivity, specificity and confusion
matrix are used. To further assess the performance of the ViT model, we trained the
pre-trained Resnet18 using our fused images.
3.2. Result
Table 1 showed the outcome of the structural similarity measurement of the original
MRI, original PET, and fused image of AD, EMCI, and LMCI participants. As shown in
Table 2, the model proposed for the validation accuracy results and the CNN model for the
AD/EMCI classification, as well as the AD/LMCI classification, according to the validation
set, are 98.50%, 98.59%, 94.03%, and 95.00%, respectively. The training/validation loss
curve and the accuracy curve for AD/LMCI classification for our proposed model and CNN
model is shown in Figure 4 and Figure 5, respectively. The results of our proposed method
on multimodal data from MRI and PET image are obviously better than the proposed
model on single modal data for all experiments. Our proposed model was evaluated on the
dataset of new participants from the ADNI database, and the result is depicted in Table 3.
The predictive analytics of our proposed model for AD/EMCI ( MRI test data), AD/EMCI
( PET test data), AD/LMCI (MRI test data), and AD/LMCI ( PET test data) is presented
using a confusion matrix, as shown in Figure 6.
Table 1. Similarity score of original MRI and PET images.
Modality Similarity Score
MRI (AD) 0.779
PET (AD) 0.812
MRI (EMCI) 0.720
PET (EMCI) 0.840
MRI (LMCI) 0.702
PET( LMCI) 0.890
Table 2. Training precision of the proposed model and pre-trained CNN using PET, MRI, and fused
data (MRI + PET).
Group MRI PET Fused (MRI+PET)
AD/EMCI
(Proposed)
98.1% 97.09% 98.50%
AD/LMCI
(Proposed)
96.11% 94.70% 99.58%
AD/EMCI
(Pre-trained CNN)
92.40% 93.56% 94.03%
AD/LMCI
(Pre-trained CNN)
93.33% 93.56% 95.00%
11. Electronics 2023, 12, 1218 11 of 19
Table 3. Accuracy of the proposed model on test data.
Group MRI PET
AD/EMCI 81.25% 93.75%
AD/LMCI 81.25% 93.75%
Figure 4. Training accuracy, validation accuracy, training loss, and validation loss for the proposed
ViT model for AD/LMCI classification.
Figure 5. Training accuracy, validation accuracy, training loss, and validation loss for pre-trained
ResNet18 model for AD/LMCI.
From Table 1, the structural similarity of original MRI, original PET, and fused image
gave a score of above 0.750 in all the experiments.A performance analysis of SIMM using
different imaging fusion techniques was carried out, and the result analysis shows the
lowest score of 0.69 and the highest score of 0.97 for the MRI dataset among nine different
experiments [52]. Information in visible and infrared spectra were fused, and the quality of
the fused image was evaluated using SIMM, which gave 0.748 [53]. The result from Table 1
indicates that the original MRI, original PET, and the fused images have a high level of
structural similarity. A score that is above 0.750 is relatively high and indicates that the
two images have a strong visual resemblance in terms of the overall structure and layout
of the features within the images. This suggests that the fusion process was successful in
maintaining the integrity of the original MRI image while adding additional information
from other sources.
The result of Table 2 means that the accuracy of the model when using only MRI data
is 98.14%, when using only PET data, it is 97.09%, and when using merged MRI and PET
data, it is 98.50% for the AD/EMCI classification. The use of MRI alone may result in an
accurate classification, but it may not be able to identify specific pathological changes in
the brain linked to AD/EMCI, which could be detected using PET imaging. When the two
imaging methods are combined, they can provide a more complete picture of the brain,
potentially improving the accuracy of classification. Even a slight improvement in accuracy
achieved through the fusion of both modalities may have clinical significance, leading to a
12. Electronics 2023, 12, 1218 12 of 19
more precise and earlier diagnosis and treatment of AD/EMCI. The result of Table 3 shows
that the precision of the multimodal fusion model, when tested with MRI data, is lower
than when tested with PET data. One major reason could be that the optimisation method
used for the fusion could not effectively combine the information from both modalities,
as in the case of MRI. The fused MRI and PET data seem to have the highest accuracy,
indicating that combining both types of data may improve the model’s performance. It is
important to note that these results are specific to the ADNI dataset and may not necessarily
apply to other datasets or scenarios for AD/LMCI classification. Figure 6a indicates that,
of the total of 16 samples, six were correctly classified as AD (true positive), two were
incorrectly classified as EMCI (false positive), seven were correctly classified as EMCI (true
negative), and one was incorrectly classified as AD (false negative) for MRI test data.
(a) (b)
(c) (d)
Figure 6. Confusion matrices. (a) Confusion matrix for AD/EMCI (MRI test data), (b) Confusion
matrix for AD/EMCI (PET test data), (c) Confusion matrix for AD/LMCI (MRI test data), and
(d) Confusion matrix for AD/LMCI (PET test data).
The confusion matrix for testing the proposed model on PET test data is shown in
Figure 6b, which is similar to the MRI test data, only that there were no false positive results,
which means that the model correctly identified eight cases of AD. The result shows that
the proposed model performed well on PET test data from ADNI than MRI test data with a
high accuracy and precision of 100%, as well as a relatively high recall of 88.89%. Figure 6c
means that all eight instances of AD were correctly identified. There were no false positive
cases where AD was incorrectly identified. However, there were three false negative
cases of LMCI on MRI test data while on PET test data, only one instance was incorrectly
classified as LMCI. Generally, the proposed model gave a high specificity of 100%, and
88.00% in identifying LMCI and EMCI from PET test data, respectively. Additionally, the
proposed model achieved specificity of 62.50% and 87.50% in identifying LMCI and EMCI
from MRI test data, respectively. In this case, LMCI and EMCI are true negatives (TNs),
and the proportion of TN results among all the patients that the model classifies as LMCI
or EMCI is the specificity. The model is tested only on 16 samples for AD vs LMCI and AD
vs EMCI because of the limited availability of data for LMCI and EMCI groups. However,
13. Electronics 2023, 12, 1218 13 of 19
this is sufficient to help us make informed decisions about whether further improvements
need to be made to the model before it is used in practice. The proposed model performed
better on PET test data, which could be an indication that the percentage of the functional
information in the fused image is higher than the structural information. This is due to
the imaging modalities and their respective signal-to-noise ratios, image resolution, and
contrast. The fusion approach may perform better on one modality compared to the other.
3.3. Visualization
Transformer attribution is utilised to generate desired visualisation to understand the
decision-making process of the proposed model. The generated maps for the AD fused
image (the top left image) left) and the EMCI fused image (the top right image) right) are
depicted in Figure 7. The bottom left image shows the map of the region to classify AD, and,
at the bottom right, there is an image showing the map of the region for EMCI classification.
The major fact we wanted to verify is to see which regions of the images (structural and
functional) the model focused on the most to make the final diagnosis.
From Figure 7, the proposed model used more functional information for the AD/EMCI
binary classification.
3.4. Comparison with Existing Methods
The proposed multimodal fusion model is compared with some existing models to
demonstrate its effectiveness for early diagnosis of AD (Table 4). This followed binary
classification of AD/EMCI using multimodal image data from MRI and PET or any other
multimodal imaging fusion consisting of structural and functional information. Forouzan-
nezhad et al. [36] combined MRI, as well as PET for AD/EMCI classification using a
deep neural network with 83.2% precision. Forouzannezhad et al. [37] used a probabilis-
tic method to collect the most relevant features from MRI, PET, and DTI for six binary
classifications. The authors provided a distinction of the early and late stages of MCI
using the Gaussian process and integrating the Bayesian prediction. Meng et al. [54] used
DTI and fMRI to predict the diseased brain regions associated with AD, and a neural net-
work using Lasso regression was used for the extraction and classification of multimodal
image features.
Figure 7. Relevancy Maps AD and EMCI.
14. Electronics 2023, 12, 1218 14 of 19
Table 4. The comparison results of EMCI vs AD classification with existing techniques.
References Method Modality Accuracy
[36] Deep Neural Network MRI + PET + 83.20%
[37] Gaussian process MRI + PET + DTI 88.10%
[54] LassoNet + Neural network DTI +fMRI 85.00%
Proposed Model Pretrained ViT + pixel image fusion MRI +PET (PET test data) 93.75%
Proposed Model Pretrained ViT + pixel image fusion MRI +PET (MRI test data) 81.25%
4. Discussion
4.1. Answers to Research Questions
4.1.1. Answer to Research Question 1
The research question aimed to determine how MRI and PET data could be effectively
combined for the early detection of AD. To address this, the authors of the paper proposed
a multimodal fusion approach based on discrete wavelet transform (DWT) optimized with
transfer learning based on the VGG16 model.
The proposed approach involves preprocessing the MRI and PET images to ensure
they are properly aligned and then combining the images using DWT to create a fused
image. The purpose of the fusion is to merge the complementary information present in
the MRI and PET data to increase the precision of diagnosing AD. MRI provides structural
information about the brain, while PET data provide functional information, and the fusion
of these modalities can provide a more comprehensive picture of the brain’s health.
4.1.2. Answer to Research Question 2
The second research question in this paper aimed to evaluate the performance of the
proposed multimodal fusion approach based on a vision transformer (ViT).
To answer this question, the authors performed experiments on the AD Neuroimaging
Initiative (ADNI) dataset, which includes MRI and PET images from participants with
different stages of AD. The fused images were then fed into a pre-trained vision transformer,
which was fine-tuned to classify the images into either AD/EMCI or AD/LMCI categories.
The results showed that the ViT model achieved an accuracy of 81.25% on the MRI test
data and 93.75% on the PET test data (see Table 3). These results suggest that the proposed
multimodal fusion approach is effective in extracting important information from the MRI
and PET data and using it to classify the images into different stages of AD.
The results suggest that the ViT model is a promising approach for analyzing multi-
modal neuroimaging data for early detection of AD. The high accuracy achieved by the
model provides evidence that the proposed approach is effective in combining information
from different neuroimaging modalities and using it to accurately diagnose AD.
4.1.3. Answer to Research Question 3
The third research question in this paper aimed to compare the performance of the
proposed multimodal fusion approach based on a vision transformer (ViT) with other
existing multimodal fusion models for AD diagnosis.
To answer this question, the authors compared the performance of their proposed ViT
model with other existing multimodal fusion models on the AD Neuroimaging Initiative
(ADNI) dataset. The dataset includes MRI and PET images from participants with different
stages of AD.
The results showed that the ViT model outperformed the other existing multimodal
fusion models in terms of accuracy for the diagnosis of AD using PET data. The ViT model
achieved an accuracy of 93.75% on the PET test data, while the other existing models
achieved a lower accuracy (see Table 4).
These results suggest that the ViT model with multimodal data fusion is a promising
approach to the diagnosis of AD, especially compared to other existing multimodal fusion
models. The ability of the proposed model to effectively extract information from the fused
15. Electronics 2023, 12, 1218 15 of 19
MRI and PET images and use it for an accurate diagnosis sets it apart from other existing
models and makes it a valuable tool for early detection of AD.
4.1.4. Answer to Research Question 4
The fourth research question in this paper aimed to evaluate the generalisability of
the proposed multimodal fusion approach based on a vision transformer (ViT) for the
diagnosis of AD.
To answer this question, the authors evaluated the performance of their proposed ViT
model on a different test dataset of MRI and PET images from participants with different
stages of AD. This was done to test the generalization capability of the model and to see if
it can perform well on unseen data.
The results showed that the proposed multimodal fusion approach achieved a high
level of precision, 81.25%, on the MRI test data and 93.75% on the PET test data. These
results indicate that the model has the ability to generalise well to new unseen data, which
is a crucial aspect of its practical application in real-world settings.
These results suggest that the proposed multimodal fusion approach is a promising
approach for the diagnosis of AD, as it can generalize well to new unseen data. This
capability of the model is important for its practical use in real-world settings, where it may
be applied to diagnose AD in new participants who were not present in the training dataset.
4.1.5. Answer to Research Question 5
The fifth research question in this paper aimed to identify the limitations and potential
improvements of the proposed multimodal fusion approach and the use of ViT for medical
imaging data analysis.
The limitations of the proposed multimodal fusion approach include the potential loss
of unique features of each modality in the fused image and the need for further testing to
ensure the accuracy of the model. Furthermore, the fusion parameters could be optimized
in the future using weighting factors to improve the performance of the fused image.
Potential improvements of the proposed approach could include the integration of
additional imaging modalities, the optimization of fusion parameters, and the development
of more advanced deep learning models, such as attention-based models. Furthermore,
incorporating additional clinical and demographic data into the model could also improve
its performance. The use of ViT for medical imaging data analysis could also benefit
from further fine-tuning on larger and more diverse datasets, as well as the integration of
additional pre-processing techniques, such as segmentation and registration.
4.2. Limitations
The limitations of this study include:
• Limited dataset: the study was carried out on a limited dataset from the Alzheimer’s
Disease Neuroimaging Initiative (ADNI) database, and the results may not be gener-
alised to larger or diverse datasets.
• Fusion parameters: the fusion parameters in the study were not optimised to their full
potential, and further optimisation may be necessary to improve the performance of
the fused image.
• Single-mode performance: the performance of the model using single modalities
(MRI or PET) was not evaluated, so it is not clear how well the model would perform
without the fusion of data.
• Limitations of ViT: the use of a ViT model for the analysis of medical imaging data is
still a relatively new area of research, and its limitations have not been fully explored.
The limitations associated with the methodology used in this paper can be stated
as follows:
• Fusion technique: the fusion technique used in the study (DWT) may not be optimal
for all types of medical imaging data, and other fusion techniques should be evaluated.
16. Electronics 2023, 12, 1218 16 of 19
• Transfer learning: the study relied on transfer learning with a pre-trained VGG16 model,
and the results may not generalise to other types of pre-training or architectures.
• Model selection: the selection of a ViT model for the study was based on its perfor-
mance on a different task, and the suitability of ViT for the task of AD classification
has not been fully established.
• These limitations highlight the need for further research and evaluation of the pro-
posed multimodal fusion approach and the use of ViT for the analysis of medical
imaging data.
5. Conclusions
In this study, a multimodal fusion model based on fine-tuned ViT is proposed for the
binary classification of AD stages (AD/EMCI and AD/LMCI). MRI and PET images are
fused with adequate preprocessing and image registration. To confirm the generalisation
of our proposed multimodal fused image model, the proposed model was evaluated
on different test data of MRI and PET from participants from the ADNI database with
81.25% and 93.75%, respectively. This suggests that the proposed ViT model, which
was trained with fused data, can achieve a high level of accuracy when tested on ADNI
PET test data. This implies that the model has acquired knowledge to effectively extract
important characteristics of the fused data and is able to generalise well to new unseen
data. Furthermore, the fact that the model achieved a 93.75% accuracy suggests that it can
accurately classify a significant proportion of the PET test data. Overall, this indicates that
the proposed ViT model is a promising approach for classifying AD stages when trained
on fused MRI and PET data, in which classification with PET data gave better accuracy
than MRI test data. The proposed model performed significantly better on PET data, but
further research is required to improve its accuracy in MRI data classification.
The comparative analysis with existing multimodal models indicated that the pro-
posed model considerably outperforms the others in generalizing well on PET data with
an accuracy of 93.75%. The use of transfer learning for image interpolation offered a high
quality fused image. The suggested diagnosis approach assists in validating observations
and spotting early anomalies, fulfilling the needs of real-world applications. The use of
fused image to train our proposed model for AD classification helps to reduce the cost
of building different models for different imaging modalities, and this can save time and
resources that would otherwise be spent building and training separate models. By demon-
strating the feasibility and potential benefits of MRI-PET fusion, the combination of other
imaging modalities, such as diffusion-weighted imaging or functional MRI, could also be
investigated in the future. Other visualization techniques, such as Gradcam, could also be
considered in future work to understand the proposed model decision-making. Evaluation
of the impact of image quality on fused data is another future direction in this research.
Author Contributions: Conceptualization, R.M.; methodology, R.M.; software, M.O.; validation,
M.O., R.M. and R.D.; formal analysis, M.O., R.M. and R.D.; investigation, M.O., R.M. and R.D.;
resources, R.M.; data curation, M.O.; writing—original draft preparation, M.O. and R.M.; writ-
ing—review and editing, R.D.; visualization, M.O.; supervision, R.M. All authors have read and
agreed to the published version of the manuscript.
Funding: This research received no external funding.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: The ADNI database is available from http://adni.loni.usc.edu/ (ac-
cessed 23 December 2021).
Acknowledgments: The authors thank esteemed Rb. Herbert von Allzenbutt for his thoughtful
remarks on the medical analysis of the dark cavity in the fMRI data.
Conflicts of Interest: The authors declare no conflict of interest.
17. Electronics 2023, 12, 1218 17 of 19
References
1. DeTure, M.A.; Dickson, D.W. The neuropathological diagnosis of Alzheimer’s disease. Mol. Neurodegener. 2019, 14, 32. [CrossRef]
2. Piovezan, R.D.; Oliveira, D.; Arias, N.; Acosta, D.; Prince, M.J.; Ferri, C.P. Mortality Rates and Mortality Risk Factors in Older
Adults with Dementia from Low- and Middle-Income Countries: The 10/66 Dementia Research Group Population-Based Cohort
Study. J. Alzheimer’s Dis. 2020, 75, 581–593. [CrossRef]
3. Gaugler, J.E.; James, B.; Johnson, T.; Reimer, J.; Solis, M.; Weuve, J.; Buckley, R.F.; Hohman, T.J. 2022 Alzheimer’s disease facts and
figures. Alzheimer’s Dement. 2022, 18, 700–789. [CrossRef]
4. Odusami, M.; Maskeliunas, R.; Damaševičius, R.; Misra, S. Comparable Study of Pre-Trained Model on Alzheimer Disease Classification;
Lecture Notes in Computer Science Volume 12953; Springer: Berlin/Heidelberg, Germany, 2021; pp. 63–74.
5. Li, H.; Habes, M.; Wolk, D.A.; Fan, Y. A deep learning model for early prediction of Alzheimer’s disease dementia based on
hippocampal magnetic resonance imaging data. Alzheimer’s Dement. 2019, 15, 1059–1070. [CrossRef]
6. Bartos, A.; Gregus, D.; Ibrahim, I.; Tintěra, J. Brain volumes and their ratios in Alzheimer´s disease on magnetic resonance
imaging segmented using Freesurfer 6.0. Psychiatry Res. Neuroimaging 2019, 287, 70–74. [CrossRef]
7. Chandra, A.; Dervenoulas, G.; Politis, M. Magnetic resonance imaging in Alzheimer’s disease and mild cognitive impairment.
J. Neurol. 2018, 266, 1293–1302. [CrossRef]
8. Holiga, S.; Abdulkadir, A.; Klöppel, S.; Dukart, J. Functional Magnetic Resonance Imaging in Alzheimer’ Disease Drug
Development. In Biomarkers for Alzheimer’s Disease Drug Development; Springer: Berlin/Heidelberg, Germany, 2018; pp. 159–163.
[CrossRef]
9. Forouzannezhad, P.; Abbaspour, A.; Fang, C.; Cabrerizo, M.; Loewenstein, D.; Duara, R.; Adjouadi, M. A survey on applications
and analysis methods of functional magnetic resonance imaging for Alzheimer’s disease. J. Neurosci. Methods 2019, 317, 121–140.
[CrossRef]
10. Wang, X.H.; Zhao, J.; Du, Y.H.; Ding, X.T.; Men, G.Z. Alteration of functional connectivity in patients with Alzheimer’s disease
revealed by resting-state functional magnetic resonance imaging. Neural Regen. Res. 2020, 15, 285. [CrossRef]
11. Ossenkoppele, R.; Rabinovici, G.D.; Smith, R.; Cho, H.; Schöll, M.; Strandberg, O.; Palmqvist, S.; Mattsson, N.; Janelidze, S.;
Santillo, A.; et al. Discriminative Accuracy of Positron Emission Tomography for Alzheimer Disease vs Other Neurodegenerative
Disorders. JAMA 2018, 320, 1151. [CrossRef]
12. Guo, J.; Qiu, W.; Li, X.; Zhao, X.; Guo, N.; Li, Q. Predicting Alzheimer’s Disease by Hierarchical Graph Convolution from Positron
Emission Tomography Imaging. In Proceedings of the 2019 IEEE International Conference on Big Data (Big Data), Los Angeles,
CA, USA, 9–12 December 2019; IEEE: New York, NY, USA, 2019. [CrossRef]
13. Velazquez, M.; Anantharaman, R.; Velazquez, S.; Lee, Y.; Alzheimer’s Disease Neuroimaging Initiative. RNN-Based Alzheimer’s
Disease Prediction from Prodromal Stage using Diffusion Tensor Imaging. In Proceedings of the 2019 IEEE International
Conference on Bioinformatics and Biomedicine (BIBM), San Diego, CA, USA, 18–21 November 2019; pp. 1665–1672.
14. Amoroso, N.; Rocca, M.L.; Bellotti, R.; Fanizzi, A.; Monaco, A.; Tangaro, S. Alzheimer’s disease diagnosis based on the
Hippocampal Unified Multi-Atlas Network (HUMAN) algorithm. BioMedical Eng. Online 2018, 17, 6. [CrossRef]
15. Lian, C.; Liu, M.; Zhang, J.; Shen, D. Hierarchical Fully Convolutional Network for Joint Atrophy Localization and Alzheimer’s
Disease Diagnosis Using Structural MRI. IEEE Trans. Pattern Anal. Mach. Intell. 2020, 42, 880–893. [CrossRef]
16. Gupta, Y.; Lee, K.H.; Choi, K.Y.; Lee, J.J.; Kim, B.C.; Kwon, G.R. Early diagnosis of Alzheimer’s disease using combined features
from voxel-based morphometry and cortical, subcortical, and hippocampus regions of MRI T1 brain images. PLoS ONE 2019,
14, e0222446. [CrossRef]
17. Toshkhujaev, S.; Lee, K.H.; Choi, K.Y.; Lee, J.J.; Kwon, G.R.; Gupta, Y.; Lama, R.K. Classification of Alzheimer’s Disease and Mild
Cognitive Impairment Based on Cortical and Subcortical Features from MRI T1 Brain Images Utilizing Four Different Types of
Datasets. J. Healthc. Eng. 2020, 2020, 1–14. [CrossRef]
18. Hossain, M.S.; Kimura, F.; Yagi, Y.; Yamaguchi, M.; Nakamura, T. Practical image quality evaluation for whole slide imaging
scanner. In Proceedings of the Biomedical Imaging and Sensing Conference, Yokohama, Japan, 25–27 April 2018; Matoba, O.,
Awatsuji, Y., Yatagai, T., Aizu, Y., Eds.; SPIE: Bellingham, WA, USA, 2018. [CrossRef]
19. Bi, X.; Jiang, Q.; Sun, Q.; Shu, Q.; Liu, Y. Analysis of Alzheimer’s Disease Based on the Random Neural Network Cluster in fMRI.
Front. Neuroinformatics 2018, 12. [CrossRef]
20. Hojjati, S.H.; Ebrahimzadeh, A.; Khazaee, A.; Babajani-Feremi, A. Predicting conversion from MCI to AD by integrating rs-fMRI
and structural MRI. Comput. Biol. Med. 2018, 102, 30–39. [CrossRef]
21. Wang, Y.; Li, C. Functional magnetic resonance imaging classification based on random forest algorithm in Alzheimer’s disease.
In Proceedings of the 2019 International Conference on Image and Video Processing, and Artificial Intelligence, Shanghai, China,
23–25 August 2019; Su, R., Ed.; SPIE: Bellingham, WA, USA, 2019. [CrossRef]
22. Marchitelli, R.; Aiello, M.; Cachia, A.; Quarantelli, M.; Cavaliere, C.; Postiglione, A.; Tedeschi, G.; Montella, P.; Milan, G.; Salvatore,
M.; et al. Simultaneous resting-state FDG-PET/fMRI in Alzheimer Disease: Relationship between glucose metabolism and
intrinsic activity. NeuroImage 2018, 176, 246–258. [CrossRef]
23. Chen, D.; Jiang, J.; Lu, J.; Wu, P.; Zhang, H.; Zuo, C.; Shi, K. Brain Network and Abnormal Hemispheric Asymmetry Analyses to
Explore the Marginal Differences in Glucose Metabolic Distributions Among Alzheimer’s Disease, Parkinson’s Disease Dementia,
and Lewy Body Dementia. Front. Neurol. 2019, 10. [CrossRef]
18. Electronics 2023, 12, 1218 18 of 19
24. Kadry, S.; Damasevicius, R.; Taniar, D.; Rajinikanth, V.; Lawal, I.A. Extraction of Tumour in Breast MRI using Joint Thresh-
olding and Segmentation—A Study. In Proceedings of the 2021 IEEE 7th International Conference on Bio Signals, Images and
Instrumentation, Chennai, India, 25–27 March 2021.
25. Rajinikanth, V.; Kadry, S.; Nam, Y. Convolutional-neural-network assisted segmentation and svm classification of brain tumor in
clinical mri slices. Inf. Technol. Control 2021, 50, 342–356. [CrossRef]
26. Khan, M.A.; Khan, A.; Alhaisoni, M.; Alqahtani, A.; Alsubai, S.; Alharbi, M.; Malik, N.A.; Damaševičius, R. Multimodal brain
tumor detection and classification using deep saliency map and improved dragonfly optimization algorithm. Int. J. Imaging Syst.
Technol. 2022, Early View. [CrossRef]
27. Rajinikanth, V.; Kadry, S.; Damasevicius, R.; Sujitha, R.A.; Balaji, G.; Mohammed, M.A. Glioma/Glioblastoma Detection in Brain
MRI using Pre-trained Deep-Learning Scheme. In Proceedings of the 2022 3rd International Conference on Intelligent Computing,
Instrumentation and Control Technologies: Computational Intelligence for Smart Systems, Kannur, Kerala, 11–12 August 2022;
pp. 987–990.
28. Badjie, B.; Deniz Ülker, E. A Deep Transfer Learning Based Architecture for Brain Tumor Classification Using MR Images. Inf.
Technol. Control 2022, 51, 332–344. [CrossRef]
29. Odusami, M.; Maskeliūnas, R.; Damaševičius, R.; Misra, S. ResD Hybrid Model Based on Resnet18 and Densenet121 for Early
Alzheimer Disease Classification; Lecture Notes in Networks and Systems Volume 418; Springer: Berlin/Heidelberg, Germany, 2022;
pp. 296–305.
30. Ramya, J.; Maheswari, B.U.; Rajakumar, M.P.; Sonia, R. Alzheimer’s Disease Segmentation and Classification on MRI Brain
Images Using Enhanced Expectation Maximization Adaptive Histogram (EEM-AH) and Machine Learning. Inf. Technol. Control
2022, 51, 786–800. [CrossRef]
31. Venugopalan, J.; Tong, L.; Hassanzadeh, H.R.; Wang, M.D. Multimodal deep learning models for early detection of Alzheimer’s
disease stage. Sci. Rep. 2021, 11, 3254. [CrossRef] [PubMed]
32. Sarraf, S.; Tofighi, G. Classification of Alzheimer’s Disease Structural MRI Data by Deep Learning Convolutional Neural
Networks. arXiv 2016, arXiv:1607.06583. [CrossRef].
33. Abdelaziz, M.; Wang, T.; Elazab, A. Alzheimer’s disease diagnosis framework from incomplete multimodal data using
convolutional neural networks. J. Biomed. Inform. 2021, 121, 103863. [CrossRef]
34. Jin, L.; Zhao, K.; Zhao, Y.; Che, T.; Li, S. A Hybrid Deep Learning Method for Early and Late Mild Cognitive Impairment
Diagnosis With Incomplete Multimodal Data. Front. Neuroinformatics 2022, 16, 843566. [CrossRef]
35. Khagi, B.; Kwon, G.R. 3D CNN Design for the Classification of Alzheimer’s Disease Using Brain MRI and PET. IEEE Access 2020,
8, 217830–217847. [CrossRef]
36. Forouzannezhad, P.; Abbaspour, A.; Li, C.; Cabrerizo, M.; Adjouadi, M. A Deep Neural Network Approach for Early Diagnosis of
Mild Cognitive Impairment Using Multiple Features. In Proceedings of the 2018 17th IEEE International Conference on Machine
Learning and Applications (ICMLA), Orlando, FL, USA, 17–20 December 2018; IEEE: New York, NY, USA, 2018. [CrossRef]
37. Forouzannezhad, P.; Abbaspour, A.; Li, C.; Fang, C.; Williams, U.; Cabrerizo, M.; Barreto, A.; Andrian, J.; Rishe, N.; Curiel,
R.E.; et al. A Gaussian-based model for early detection of mild cognitive impairment using multimodal neuroimaging. J. Neurosci.
Methods 2020, 333, 108544. [CrossRef] [PubMed]
38. Aqeel, A.; Hassan, A.; Khan, M.A.; Rehman, S.; Tariq, U.; Kadry, S.; Majumdar, A.; Thinnukool, O. A Long Short-Term Memory
Biomarker-Based Prediction Framework for Alzheimer’s Disease. Sensors 2022, 22, 1475. [CrossRef]
39. Sarraf, S.; Sarraf, A.; DeSouza, D.D.; Anderson, J.A.E.; Kabia, M. OViTAD: Optimized Vision Transformer to Predict Various Stages
of Alzheimer’s Disease Using Resting-State fMRI and Structural MRI Data. bioXiv 2021. [CrossRef]
40. Xing, X.; Liang, G.; Zhang, Y.; Khanal, S.; Lin, A.L.; Jacobs, N. Advit: Vision Transformer On Multi-Modality Pet Images For
Alzheimer Disease Diagnosis. In Proceedings of the 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI),
Kolkata, India, 28–31 March 2022. [CrossRef]
41. Kushol, R.; Masoumzadeh, A.; Huo, D.; Kalra, S.; Yang, Y.H. Addformer: Alzheimer’s Disease Detection from Structural Mri
Using Fusion Transformer. In Proceedings of the 2022 IEEE 19th International Symposium on Biomedical Imaging (ISBI), Kolkata,
India, 28–31 March 2022. [CrossRef]
42. Pan, J.; Wang, S. Cross-Modal Transformer GAN: A Brain Structure-Function Deep Fusing Framework for Alzheimer’s Disease.
arXiv 2022, arXiv:2206.13393. [CrossRef].
43. Muzammil, S.R.; Maqsood, S.; Haider, S.; Damaševičius, R. CSID: A novel multimodal image fusion algorithm for enhanced
clinical diagnosis. Diagnostics 2020, 10, 904. [CrossRef]
44. Andreella, A.; Finos, L. Procrustes Analysis for High-Dimensional Data. Psychometrika 2022, 87, 1422–1438. [CrossRef] [PubMed]
45. Dwivedi, S.; Goel, T.; Tanveer, M.; Murugan, R.; Sharma, R. Multimodal Fusion-Based Deep Learning Network for Effective
Diagnosis of Alzheimer’s Disease. IEEE MultiMedia 2022, 29, 45–55. [CrossRef]
46. Grobbelaar, M.; Phadikar, S.; Ghaderpour, E.; Struck, A.F.; Sinha, N.; Ghosh, R.; Ahmed, M.Z.I. A Survey on Denoising Techniques
of Electroencephalogram Signals Using Wavelet Transform. Signals 2022, 3, 577–586. [CrossRef]
47. Khmag, A. Additive Gaussian noise removal based on generative adversarial network model and semi-soft thresholding
approach. Multimed. Tools Appl. 2023, 82, 7757–7777. [CrossRef]
48. Chen, Z.; Xie, L.; Niu, J.; Liu, X.; Wei, L.; Tian, Q. Visformer: The Vision-friendly Transformer. arXiv 2021, arXiv:2104.12533.
[CrossRef].
19. Electronics 2023, 12, 1218 19 of 19
49. Khan, S.; Naseer, M.; Hayat, M.; Zamir, S.W.; Khan, F.S.; Shah, M. Transformers in Vision: A Survey. ACM Comput. Surv. 2022,
54, 1–41. [CrossRef]
50. Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold,
G.; Gelly, S.; et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv 2020, arXiv:2010.11929.
[CrossRef].
51. Sandler, M.; Zhmoginov, A.; Vladymyrov, M.; Jackson, A. Fine-tuning Image Transformers using Learnable Memory. arXiv 2022,
arXiv:2203.15243. [CrossRef].
52. Kumar, M.; Ranjan, N.; Chourasia, B. Analysis of Medical Image Fusion Using Transform-Based Function and Neural Network.
Ann. Rom. Soc. Cell Biol. 2021, 25, 6333–6347.
53. Peng, X.; Kong, L.; Han, W.; Wang, S. Multi-Sensor Image Fusion Method for Defect Detection in Powder Bed Fusion. Sensors
2022, 22, 8023. [CrossRef] [PubMed]
54. Meng, X.; Liu, J.; Fan, X.; Bian, C.; Wei, Q.; Wang, Z.; Liu, W.; Jiao, Z. Multi-Modal Neuroimaging Neural Network-Based Feature
Detection for Diagnosis of Alzheimer’s Disease. Front. Aging Neurosci. 2022, 14, 911220. [CrossRef] [PubMed]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.