This document reviews literature on the application of generative adversarial networks (GANs) for ophthalmology image domains. GANs have been used for tasks like segmentation, data augmentation, denoising, domain transfer, and super-resolution of ophthalmic images. The review identified 48 relevant papers and found that GANs have benefited analysis of fundus photographs, optical coherence tomography scans, and other eye images. However, GANs also have limitations such as mode collapse and need clinical validation before practical use in ophthalmology. Further development could improve GAN performance for ocular image analysis and disease diagnosis.
RETINAL IMAGE CLASSIFICATION USING NEURAL NETWORK BASED ON A CNN METHODSIRJET Journal
This document discusses using a convolutional neural network to classify retinal images. Specifically, it aims to develop a system to distinguish between different retinal diseases using fundus images. The system would extract retinal features from the images like the retina, optic nerve and lesions. It then uses a CNN to detect multiple retinal diseases in fundus photographs from a structured analysis database. The CNN is trained on publicly available retinal image datasets. Neural networks have been found to effectively capture disease-specific color and texture features to enable automated diagnosis similar to human experts. The document also provides background on related work using deep learning and CNNs for tasks like lesion detection and classification of retinal diseases from fundus images.
Small overview of the startups involved in healthcare artificial intelligence, the OCT market, investments, patent and IP issues and FDA regulation.
Alternative download link: https://dl.dropboxusercontent.com/u/6757026/slideShare/retinalAI_landscape.pdf
An effective deep learning network for detecting and classifying glaucomatous...IJECEIAES
Glaucoma is a well-known complex disease of the optic nerve that gradually damages eyesight due to the increase of intraocular pressure inside the eyes. Among two types of glaucoma, open-angle glaucoma is mostly happened by high intraocular pressure and can damage the eyes temporarily or sometimes permanently, another one is angle-closure glaucoma. Therefore, being diagnosed in the early stage is necessary to safe our vision. There are several ways to detect glaucomatous eyes like tonometry, perimetry, and gonioscopy but require time and expertise. Using deep learning approaches could be a better solution. This study focused on the recognition of open-angle affected eyes from the fundus images using deep learning techniques. The study evolved by applying VGG16, VGG19, and ResNet50 deep neural network architectures for classifying glaucoma positive and negative eyes. The experiment was executed on a public dataset collected from Kaggle; however, every model performed better after augmenting the dataset, and the accuracy was between 93% and 97.56%. Among the three models, VGG19 achieved the highest accuracy at 97.56%.
Application of deep learning methods for automated analysis of retinal struct...IJECEIAES
This article examines a current area of research in the field of ophthalmology the use of deep learning methods for automated analysis of retinal structures. This work explores the use of deep learning methods such as EfficientNet and DenseNet for the automated analysis of retinal structures in ophthalmology. EfficientNet, originally proposed to balance between accuracy and computational efficiency, and DenseNet, based on dense connections between layers, are considered as tools for identifying and classifying retina features. Automated analysis includes identifying pathologies, assessing the degree of their development and, possibly, diagnosing various eye diseases. Experiments are performed on a dataset containing a variety of images of retinal structures. Results are evaluated using metrics of accuracy, sensitivity, and specificity. It is expected that the proposed deep learning methods can significantly improve the automated analysis of retinal images, which is important for the diagnosis and monitoring of eye diseases. As a result, the article highlights the significance and promise of using deep learning methods in ophthalmology for automated analysis of retinal structures. These methods help improve the early diagnosis, treatment and monitoring of eye diseases, which can ultimately lead to improved healthcare quality and improved patient lives.
From unimodal image classification to integrative multimodal deep learning pipelines in disease classification, disease management and predictive personalised healthcare.
Shap Analysis Based Gastric Cancer DetectionIRJET Journal
This document proposes a novel deep learning framework to detect gastric cancer from endoscopic images of the stomach. The framework uses a patch-based analysis where features are extracted from image patches and evaluated for cancer risk. A bag-of-features technique is then applied to the extracted features from selected patches for analysis. Experimental results show the proposed framework can effectively and efficiently detect gastric cancer from images and identify minute lesions. It achieves higher accuracy than other models using the same dataset. The framework is also more accurate than existing methods for gastric cancer detection.
Retinal Blood Vessels Exudates Classification For Detection Of Hemmorages Tha...IJSRED
This document summarizes a research paper that uses a deep learning approach with convolutional neural networks to classify pixels in retinal fundus images as either blood vessels or non-vessels. The researchers tested their technique on publicly available DRIVE dataset and achieved an average accuracy of 0.9466 and AUC of 0.9749, demonstrating the effectiveness of their deep learning approach for retinal blood vessel segmentation. Their method was inspired by previous work using deep neural networks as pixel classifiers for similar image segmentation tasks.
RETINAL IMAGE CLASSIFICATION USING NEURAL NETWORK BASED ON A CNN METHODSIRJET Journal
This document discusses using a convolutional neural network to classify retinal images. Specifically, it aims to develop a system to distinguish between different retinal diseases using fundus images. The system would extract retinal features from the images like the retina, optic nerve and lesions. It then uses a CNN to detect multiple retinal diseases in fundus photographs from a structured analysis database. The CNN is trained on publicly available retinal image datasets. Neural networks have been found to effectively capture disease-specific color and texture features to enable automated diagnosis similar to human experts. The document also provides background on related work using deep learning and CNNs for tasks like lesion detection and classification of retinal diseases from fundus images.
Small overview of the startups involved in healthcare artificial intelligence, the OCT market, investments, patent and IP issues and FDA regulation.
Alternative download link: https://dl.dropboxusercontent.com/u/6757026/slideShare/retinalAI_landscape.pdf
An effective deep learning network for detecting and classifying glaucomatous...IJECEIAES
Glaucoma is a well-known complex disease of the optic nerve that gradually damages eyesight due to the increase of intraocular pressure inside the eyes. Among two types of glaucoma, open-angle glaucoma is mostly happened by high intraocular pressure and can damage the eyes temporarily or sometimes permanently, another one is angle-closure glaucoma. Therefore, being diagnosed in the early stage is necessary to safe our vision. There are several ways to detect glaucomatous eyes like tonometry, perimetry, and gonioscopy but require time and expertise. Using deep learning approaches could be a better solution. This study focused on the recognition of open-angle affected eyes from the fundus images using deep learning techniques. The study evolved by applying VGG16, VGG19, and ResNet50 deep neural network architectures for classifying glaucoma positive and negative eyes. The experiment was executed on a public dataset collected from Kaggle; however, every model performed better after augmenting the dataset, and the accuracy was between 93% and 97.56%. Among the three models, VGG19 achieved the highest accuracy at 97.56%.
Application of deep learning methods for automated analysis of retinal struct...IJECEIAES
This article examines a current area of research in the field of ophthalmology the use of deep learning methods for automated analysis of retinal structures. This work explores the use of deep learning methods such as EfficientNet and DenseNet for the automated analysis of retinal structures in ophthalmology. EfficientNet, originally proposed to balance between accuracy and computational efficiency, and DenseNet, based on dense connections between layers, are considered as tools for identifying and classifying retina features. Automated analysis includes identifying pathologies, assessing the degree of their development and, possibly, diagnosing various eye diseases. Experiments are performed on a dataset containing a variety of images of retinal structures. Results are evaluated using metrics of accuracy, sensitivity, and specificity. It is expected that the proposed deep learning methods can significantly improve the automated analysis of retinal images, which is important for the diagnosis and monitoring of eye diseases. As a result, the article highlights the significance and promise of using deep learning methods in ophthalmology for automated analysis of retinal structures. These methods help improve the early diagnosis, treatment and monitoring of eye diseases, which can ultimately lead to improved healthcare quality and improved patient lives.
From unimodal image classification to integrative multimodal deep learning pipelines in disease classification, disease management and predictive personalised healthcare.
Shap Analysis Based Gastric Cancer DetectionIRJET Journal
This document proposes a novel deep learning framework to detect gastric cancer from endoscopic images of the stomach. The framework uses a patch-based analysis where features are extracted from image patches and evaluated for cancer risk. A bag-of-features technique is then applied to the extracted features from selected patches for analysis. Experimental results show the proposed framework can effectively and efficiently detect gastric cancer from images and identify minute lesions. It achieves higher accuracy than other models using the same dataset. The framework is also more accurate than existing methods for gastric cancer detection.
Retinal Blood Vessels Exudates Classification For Detection Of Hemmorages Tha...IJSRED
This document summarizes a research paper that uses a deep learning approach with convolutional neural networks to classify pixels in retinal fundus images as either blood vessels or non-vessels. The researchers tested their technique on publicly available DRIVE dataset and achieved an average accuracy of 0.9466 and AUC of 0.9749, demonstrating the effectiveness of their deep learning approach for retinal blood vessel segmentation. Their method was inspired by previous work using deep neural networks as pixel classifiers for similar image segmentation tasks.
Detection of Glaucoma using Optic Disk and Incremental Cup Segmentation from ...theijes
The document presents a new approach for detecting glaucoma using optic disk and incremental cup segmentation from retinal images. It proposes segmenting the optic disk using an automatic parameterized technique and modeling the cup region as enclosing pallor enclosed by vessel bends (r-bends). An incremental cup segmentation method uses 3D spline interpolation to derive the cup boundary considering anatomical knowledge from glaucoma experts. Results are compared to existing methods on different retinal images and show improved cup segmentation, especially in regions without depth cues where inter-observer variability is high. The approach integrates appearance and anatomical information for more accurate glaucoma assessment through optic disk and cup segmentation.
Bone age assessment based on deep learning architecture IJECEIAES
The fast advancement of technology has prompted the creation of automated systems in a variety of sectors, including medicine. One application is an automated bone age evaluation from left-hand X-ray pictures, which assists radiologists and pediatricians in making decisions about the growth status of youngsters. However, one of the most difficult aspects of establishing an automated system is selecting the best approach for producing effective and dependable predictions, especially when working with large amounts of data. As part of this work, we investigate the use of the convolutional neural networks (CNNs) model to classify the age of the bone. The work’s dataset is based on the radiological society of North America (RSNA) dataset. To address this issue, we developed and tested deep learning architecture for autonomous bone assessment, we design a new deep convolution network (DCNN) model. The assessment measures that use in this work are accuracy, recall, precision, and F-score. The proposed model achieves 97% test accuracy for bone age classification.
Framework for comprehensive enhancement of brain tumor images with single-win...IJECEIAES
Usage of grayscale format of radiological images is proportionately more as compared to that of colored one. This format of medical image suffers from all the possibility of improper clinical inference which will lead to error-prone analysis in further usage of such images in disease detection or classification. Therefore, we present a framework that offers single-window operation with a set of image enhancing algorithm meant for further optimizing the visuality of medical images. The framework performs preliminary pre-processing operation followed by implication of linear and non-linear filter and multi-level image enhancement processes. The significant contribution of this study is that it offers a comprehensive mechanism to implement the various enhancement schemes in highly discrete way that offers potential flexibility to physical in order to draw clinical conclusion about the disease being monitored. The proposed system takes the case study of brain tumor to implement to testify the framework.
Breast cancer histological images nuclei segmentation and optimized classifi...IJECEIAES
This document summarizes a research paper that proposes a deep learning method for breast cancer histological image segmentation and classification. The method uses a feature pyramid network to extract features from histological images, which are then optimized using a grasshopper optimization algorithm. The optimized features are used with a segmenting objects by locations convolutional network for nucleus segmentation. For classification, the features are passed through fully connected layers. The method achieves 88.46% segmentation accuracy and 99.2% classification accuracy on a large breast cancer image dataset. It provides an effective computer-aided approach for breast cancer analysis in a medical setting.
An Innovative Deep Learning Framework Integrating Transfer- Learning And Extr...IRJET Journal
This paper proposes a deep learning framework that uses transfer learning and an XGBoost classifier to classify breast ultrasound images. It uses a VGG16 model pre-trained on general images to extract features from ultrasound images. These features are then classified using an XGBoost classifier. On a dataset of breast ultrasound images, the approach achieved 96.7% accuracy, and precision/recall/F-scores of 100%/96%/96% for benign images, 95%/97%/96% for malignant images, and 95%/98%/97% for normal images, outperforming other automatic image classification methods.
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...IJECEIAES
The issue of brain magnetic resonance image exploration together with classification receives a significant awareness in recent years. Indeed, various computer-aided-diagnosis solutions were suggested to support radiologist in decision-making. In this circumstance, adequate image classification is extremely required as it is the most common critical brain tumors which often develop from subdural hematoma cells, which might be common type in adults. In healthcare milieu, brain MRIs are intended for identification of tumor. In this regard, various computerized diagnosis systems were suggested to help medical professionals in clinical decision-making. As per recent problems, Neuroendoscopy is the gold standard intended for discovering brain tumors; nevertheless, typical Neuroendoscopy can certainly overlook ripped growths. Neuroendoscopy is a minimally-invasive surgical procedure in which the neurosurgeon removes the tumor through small holes in the skull or through the mouth or nose. Neuroendoscopy enables neurosurgeons to access areas of the brain that cannot be reached with traditional surgery to remove the tumor without cutting or harming other parts of the skull. We focused on finding out whether or not visual images of tumor ripped lesions ended up being much better by auto fluorescence image resolution as well as narrow-band image resolution graphic evaluation jointly with the latest neuroendoscopy technique. Also, within the last several years, pathology labs began to proceed in the direction of an entirely digital workflow, using the electronic slides currently being the key element of this technique. Besides lots of benefits regarding storage as well as exploring capabilities with the image information, among the benefits of electronic slides is that they can help the application of image analysis approaches which seek to develop quantitative attributes to assist pathologists in their work. However, systems also have some difficulties in execution and handling. Hence, such conventional method needs automation. We developed and employed to look for the targeted importance along with uncovering the best-focused graphic position by way of aliasing search method incorporated with new Neuroendoscopy Adapter Module (NAM) technique.
DIABETIC RETINOPATHY DETECTION USING MACHINE LEARNING TECHNIQUEIRJET Journal
1) The document discusses a method for detecting diabetic retinal disease using integrated shallow convolutional neural networks, which can improve classification accuracy by 3% on small datasets compared to other CNN techniques.
2) It aims to classify retinal images to detect diabetic retinopathy through shallow CNNs, focusing on cases with limited labelled training data, as deep CNNs typically require large datasets for high accuracy.
3) Experimental results show the proposed approach reduces time cost to around 30% of the smallest dataset tested, which is 10% of the original dataset, while maintaining classification accuracy compared to other integrated CNN learning algorithms.
deep learning applications in medical image analysis brain tumorVenkat Projects
The tremendous success of machine learning algorithms at image recognition tasks in recent years intersects with a time of dramatically increased use of electronic medical records and diagnostic imaging. This review introduces the machine learning algorithms as applied to medical image analysis, focusing on convolutional neural networks, and emphasizing clinical aspects of the _eld. The advantage of machine learning in an era of medical big data is that signi_cant hierarchal relationships within the data can be discovered algorithmically without laborious hand-crafting of features. We cover key research areas and applications of medical image classi_cation, localization, detection, segmentation, and registration. We conclude by discussing research obstacles, emerging trends, and possible future directions.
Classification of pathologies on digital chest radiographs using machine lear...IJECEIAES
This article is devoted to the research and development of methods for classifying pathologies on digital chest radiographs using two different machine learning approaches: the eXtreme gradient boosting (XGBoost) algorithm and the deep convolutional neural network residual network (ResNet50). The goal of the study is to develop effective and accurate methods for automatically classifying various pathologies detected on chest X-rays. The study collected an extensive dataset of digital chest radiographs, including a variety of clinical cases and different classes of pathology. Developed and trained machine learning models based on the XGBoost algorithm and the ResNet50 convolutional neural network using preprocessed images. The performance and accuracy of both models were assessed on test data using quality metrics and a comparative analysis of the results was carried out. The expected results of the article are high accuracy and reliability of methods for classifying pathologies on chest radiographs, as well as an understanding of their effectiveness in the context of clinical practice. These results may have significant implications for improving the diagnosis and care of patients with chest diseases, as well as promoting the development of automated decision support systems in radiology.
Discovering Abnormal Patches and Transformations of Diabetics Retinopathy in ...cscpconf
Diabetic retinopathy (DR) is one of the retinal diseases due to long-term effect of diabetes.
Early detection for diabetic retinopathy is crucial since timely treatment can prevent
progressive loss of vision. The most common diagnosis technique of diabetic retinopathy is to
screen abnormalities through retinal fundus images by clinicians. However, limited number of
well-trained clinicians increase the possibilities of misdiagnosing. In this work, we propose a
big-data-driven automatic computer-aided diagnosing (CAD) system for diabetic retinopathy
severity regression based on transfer learning, which starts from a deep convolutional neural
network pre-trained on generic images, and adapts it to large-scale DR datasets. From images
in the training set, we also automatically segment the abnormal patches with an occlusion test,
and model the transformations and deterioration process of DR. Our results can be widely used
for fast diagnosis of DR, medical education and public-level healthcare propagation.
DISCOVERING ABNORMAL PATCHES AND TRANSFORMATIONS OF DIABETICS RETINOPATHY IN ...csandit
Diabetic retinopathy (DR) is one of the retinal diseases due to long-term effect of diabetes.Early detection for diabetic retinopathy is crucial since timely treatment can prevent
progressive loss of vision. The most common diagnosis technique of diabetic retinopathy is to screen abnormalities through retinal fundus images by clinicians. However, limited number of well-trained clinicians increase the possibilities of misdiagnosing. In this work, we propose a big-data-driven automatic computer-aided diagnosing (CAD) system for diabetic retinopathy severity regression based on transfer learning, which starts from a deep convolutional neural
network pre-trained on generic images, and adapts it to large-scale DR datasets. From images in the training set, we also automatically segment the abnormal patches with an occlusion test,and model the transformations and deterioration process of DR. Our results can be widely used for fast diagnosis of DR, medical education and public-level healthcare propagation.
DEEP FACIAL DIAGNOSIS: DEEP TRANSFER LEARNING FROM FACE RECOGNITION TO FACIAL...IRJET Journal
This document discusses using deep learning techniques like transfer learning and convolutional neural networks (CNNs) to perform computer-aided facial diagnosis of diseases from photographs. Specifically, it explores diagnosing single diseases like beta-thalassemia and multiple diseases including beta-thalassemia, hyperthyroidism, Down syndrome, and leprosy. The researchers achieve over 90% accuracy on these facial diagnosis tasks using deep transfer learning from pre-trained face recognition models, outperforming traditional machine learning methods. This shows promise for non-invasive disease screening using facial analysis with deep learning.
IRJET- Eye Diabetic Retinopathy by using Deep LearningIRJET Journal
This document discusses using deep learning techniques to detect diabetic retinopathy from retinal images. Diabetic retinopathy is a disease affecting the retina caused by complications of diabetes. It can lead to vision loss if not managed properly. The document proposes using methods like image preprocessing, feature extraction, segmentation, and classification with deep learning algorithms to automatically analyze retinal images and detect signs of diabetic retinopathy. Specifically, it involves extracting features from images using techniques like color density histograms, then training a deep learning model on those features to classify images according to the severity of diabetic retinopathy present. The goal is to enable early detection of the disease from retinal images to help prevent vision loss.
This document describes a proposed method for classifying chest x-ray images to diagnose lung infections using convolutional neural networks (CNNs). The objectives are to examine if transfer learning from different source domains can improve performance for classifying healthy, pneumonia and COVID-19 cases using a small dataset. The proposed methodology includes collecting datasets, training a CNN model using transfer learning, evaluating performance using a confusion matrix, and identifying opportunities for future enhancement like exploring different network architectures and domains.
Generalized deep learning model for Classification of Gastric, Colon and Rena...IRJET Journal
This document proposes developing a generalized deep learning model to classify gastric, colon, and renal cancer using a single model. The model would be trained on whole slide images of tissue samples fed through an EfficientNet model pre-trained on ImageNet. The model would be trained using transfer learning with partial transfusion to demonstrate the ability to classify pathology images from different sites. Previous studies have developed models to classify individual tissue types but not a unified model. The proposed model aims to address situations where the tissue site of origin is unknown.
Brain Tumor Detection From MRI Image Using Deep LearningIRJET Journal
This document presents a study on using deep learning techniques for brain tumor detection from MRI images. It proposes two Convolutional Neural Network models - one without transfer learning that achieves 81.42% accuracy, and one with transfer learning using the VGG16 architecture that achieves significantly higher accuracy of 98.8%. The study uses a dataset of over 5,000 MRI images categorized as normal, benign tumor, or malignant tumor. Data preprocessing techniques like filtering and enhancement are applied before training the models. Transfer learning helps reduce training time and improves model performance for tumor classification compared to training from scratch without transferring learned features.
This document describes a study that used deep learning models to diagnose Alzheimer's disease using MRI scans. Specifically, it used convolutional neural networks (CNNs) trained on MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The study achieved an F1-score of 89% for classifying Alzheimer's vs normal cognition. It also used a CycleGAN for data augmentation, which generated additional synthetic MRI images and improved the CNN classification accuracy to 95% F1-score. The document provides background on Alzheimer's diagnosis, CNNs, GANs including CycleGAN, related work applying machine learning to Alzheimer's diagnosis, and the methodology used in this study for data preprocessing, model training, and evaluating the impact of GAN
APPLICATION OF CNN MODEL ON MEDICAL IMAGEIRJET Journal
The document discusses using convolutional neural network (CNN) models to detect diseases from medical images such as chest X-rays. It describes how CNN models can be trained on large labeled datasets of chest X-rays to learn patterns and features that indicate diseases. The document then evaluates several CNN architectures - including VGG-16, ResNet, DenseNet, and InceptionNet - for classifying chest X-rays as normal or infected. It finds these models achieve high accuracy, with metrics like accuracy over 89% and AUC over 0.94. In conclusion, deep learning models show promising results for automated disease detection from medical images.
Robustness in Machine Learning Explanations Does It Matter-1.pdfDaniel983829
This document discusses the desirability of robustness in machine learning explanations. It argues that robustness is desirable to the extent we want explanations to reflect real patterns in the data and the world. The importance of explanations capturing real patterns depends on the problem context. In some contexts, non-robust explanations can pose moral hazards. Clarifying how much we want explanations to capture real patterns can help determine whether the "Rashomon effect" is beneficial or problematic.
Robustness and Regularization of Support Vector Machines.pdfDaniel983829
This document summarizes a research paper that establishes an equivalence between robust optimization and regularization as applied to support vector machines (SVMs). It shows that regularized SVMs can be formulated as a robust optimization problem that minimizes error under potential adversarial noise or disturbances in the training data. This provides an alternative explanation for why regularization improves generalization performance of SVMs. The document also discusses implications for algorithm development and analysis, including a new proof of consistency for SVMs based on their robustness interpretation.
More Related Content
Similar to Application of generative adversarial networks (GAN) for ophthalmology image domains - a survey.pdf
Detection of Glaucoma using Optic Disk and Incremental Cup Segmentation from ...theijes
The document presents a new approach for detecting glaucoma using optic disk and incremental cup segmentation from retinal images. It proposes segmenting the optic disk using an automatic parameterized technique and modeling the cup region as enclosing pallor enclosed by vessel bends (r-bends). An incremental cup segmentation method uses 3D spline interpolation to derive the cup boundary considering anatomical knowledge from glaucoma experts. Results are compared to existing methods on different retinal images and show improved cup segmentation, especially in regions without depth cues where inter-observer variability is high. The approach integrates appearance and anatomical information for more accurate glaucoma assessment through optic disk and cup segmentation.
Bone age assessment based on deep learning architecture IJECEIAES
The fast advancement of technology has prompted the creation of automated systems in a variety of sectors, including medicine. One application is an automated bone age evaluation from left-hand X-ray pictures, which assists radiologists and pediatricians in making decisions about the growth status of youngsters. However, one of the most difficult aspects of establishing an automated system is selecting the best approach for producing effective and dependable predictions, especially when working with large amounts of data. As part of this work, we investigate the use of the convolutional neural networks (CNNs) model to classify the age of the bone. The work’s dataset is based on the radiological society of North America (RSNA) dataset. To address this issue, we developed and tested deep learning architecture for autonomous bone assessment, we design a new deep convolution network (DCNN) model. The assessment measures that use in this work are accuracy, recall, precision, and F-score. The proposed model achieves 97% test accuracy for bone age classification.
Framework for comprehensive enhancement of brain tumor images with single-win...IJECEIAES
Usage of grayscale format of radiological images is proportionately more as compared to that of colored one. This format of medical image suffers from all the possibility of improper clinical inference which will lead to error-prone analysis in further usage of such images in disease detection or classification. Therefore, we present a framework that offers single-window operation with a set of image enhancing algorithm meant for further optimizing the visuality of medical images. The framework performs preliminary pre-processing operation followed by implication of linear and non-linear filter and multi-level image enhancement processes. The significant contribution of this study is that it offers a comprehensive mechanism to implement the various enhancement schemes in highly discrete way that offers potential flexibility to physical in order to draw clinical conclusion about the disease being monitored. The proposed system takes the case study of brain tumor to implement to testify the framework.
Breast cancer histological images nuclei segmentation and optimized classifi...IJECEIAES
This document summarizes a research paper that proposes a deep learning method for breast cancer histological image segmentation and classification. The method uses a feature pyramid network to extract features from histological images, which are then optimized using a grasshopper optimization algorithm. The optimized features are used with a segmenting objects by locations convolutional network for nucleus segmentation. For classification, the features are passed through fully connected layers. The method achieves 88.46% segmentation accuracy and 99.2% classification accuracy on a large breast cancer image dataset. It provides an effective computer-aided approach for breast cancer analysis in a medical setting.
An Innovative Deep Learning Framework Integrating Transfer- Learning And Extr...IRJET Journal
This paper proposes a deep learning framework that uses transfer learning and an XGBoost classifier to classify breast ultrasound images. It uses a VGG16 model pre-trained on general images to extract features from ultrasound images. These features are then classified using an XGBoost classifier. On a dataset of breast ultrasound images, the approach achieved 96.7% accuracy, and precision/recall/F-scores of 100%/96%/96% for benign images, 95%/97%/96% for malignant images, and 95%/98%/97% for normal images, outperforming other automatic image classification methods.
Neuroendoscopy Adapter Module Development for Better Brain Tumor Image Visual...IJECEIAES
The issue of brain magnetic resonance image exploration together with classification receives a significant awareness in recent years. Indeed, various computer-aided-diagnosis solutions were suggested to support radiologist in decision-making. In this circumstance, adequate image classification is extremely required as it is the most common critical brain tumors which often develop from subdural hematoma cells, which might be common type in adults. In healthcare milieu, brain MRIs are intended for identification of tumor. In this regard, various computerized diagnosis systems were suggested to help medical professionals in clinical decision-making. As per recent problems, Neuroendoscopy is the gold standard intended for discovering brain tumors; nevertheless, typical Neuroendoscopy can certainly overlook ripped growths. Neuroendoscopy is a minimally-invasive surgical procedure in which the neurosurgeon removes the tumor through small holes in the skull or through the mouth or nose. Neuroendoscopy enables neurosurgeons to access areas of the brain that cannot be reached with traditional surgery to remove the tumor without cutting or harming other parts of the skull. We focused on finding out whether or not visual images of tumor ripped lesions ended up being much better by auto fluorescence image resolution as well as narrow-band image resolution graphic evaluation jointly with the latest neuroendoscopy technique. Also, within the last several years, pathology labs began to proceed in the direction of an entirely digital workflow, using the electronic slides currently being the key element of this technique. Besides lots of benefits regarding storage as well as exploring capabilities with the image information, among the benefits of electronic slides is that they can help the application of image analysis approaches which seek to develop quantitative attributes to assist pathologists in their work. However, systems also have some difficulties in execution and handling. Hence, such conventional method needs automation. We developed and employed to look for the targeted importance along with uncovering the best-focused graphic position by way of aliasing search method incorporated with new Neuroendoscopy Adapter Module (NAM) technique.
DIABETIC RETINOPATHY DETECTION USING MACHINE LEARNING TECHNIQUEIRJET Journal
1) The document discusses a method for detecting diabetic retinal disease using integrated shallow convolutional neural networks, which can improve classification accuracy by 3% on small datasets compared to other CNN techniques.
2) It aims to classify retinal images to detect diabetic retinopathy through shallow CNNs, focusing on cases with limited labelled training data, as deep CNNs typically require large datasets for high accuracy.
3) Experimental results show the proposed approach reduces time cost to around 30% of the smallest dataset tested, which is 10% of the original dataset, while maintaining classification accuracy compared to other integrated CNN learning algorithms.
deep learning applications in medical image analysis brain tumorVenkat Projects
The tremendous success of machine learning algorithms at image recognition tasks in recent years intersects with a time of dramatically increased use of electronic medical records and diagnostic imaging. This review introduces the machine learning algorithms as applied to medical image analysis, focusing on convolutional neural networks, and emphasizing clinical aspects of the _eld. The advantage of machine learning in an era of medical big data is that signi_cant hierarchal relationships within the data can be discovered algorithmically without laborious hand-crafting of features. We cover key research areas and applications of medical image classi_cation, localization, detection, segmentation, and registration. We conclude by discussing research obstacles, emerging trends, and possible future directions.
Classification of pathologies on digital chest radiographs using machine lear...IJECEIAES
This article is devoted to the research and development of methods for classifying pathologies on digital chest radiographs using two different machine learning approaches: the eXtreme gradient boosting (XGBoost) algorithm and the deep convolutional neural network residual network (ResNet50). The goal of the study is to develop effective and accurate methods for automatically classifying various pathologies detected on chest X-rays. The study collected an extensive dataset of digital chest radiographs, including a variety of clinical cases and different classes of pathology. Developed and trained machine learning models based on the XGBoost algorithm and the ResNet50 convolutional neural network using preprocessed images. The performance and accuracy of both models were assessed on test data using quality metrics and a comparative analysis of the results was carried out. The expected results of the article are high accuracy and reliability of methods for classifying pathologies on chest radiographs, as well as an understanding of their effectiveness in the context of clinical practice. These results may have significant implications for improving the diagnosis and care of patients with chest diseases, as well as promoting the development of automated decision support systems in radiology.
Discovering Abnormal Patches and Transformations of Diabetics Retinopathy in ...cscpconf
Diabetic retinopathy (DR) is one of the retinal diseases due to long-term effect of diabetes.
Early detection for diabetic retinopathy is crucial since timely treatment can prevent
progressive loss of vision. The most common diagnosis technique of diabetic retinopathy is to
screen abnormalities through retinal fundus images by clinicians. However, limited number of
well-trained clinicians increase the possibilities of misdiagnosing. In this work, we propose a
big-data-driven automatic computer-aided diagnosing (CAD) system for diabetic retinopathy
severity regression based on transfer learning, which starts from a deep convolutional neural
network pre-trained on generic images, and adapts it to large-scale DR datasets. From images
in the training set, we also automatically segment the abnormal patches with an occlusion test,
and model the transformations and deterioration process of DR. Our results can be widely used
for fast diagnosis of DR, medical education and public-level healthcare propagation.
DISCOVERING ABNORMAL PATCHES AND TRANSFORMATIONS OF DIABETICS RETINOPATHY IN ...csandit
Diabetic retinopathy (DR) is one of the retinal diseases due to long-term effect of diabetes.Early detection for diabetic retinopathy is crucial since timely treatment can prevent
progressive loss of vision. The most common diagnosis technique of diabetic retinopathy is to screen abnormalities through retinal fundus images by clinicians. However, limited number of well-trained clinicians increase the possibilities of misdiagnosing. In this work, we propose a big-data-driven automatic computer-aided diagnosing (CAD) system for diabetic retinopathy severity regression based on transfer learning, which starts from a deep convolutional neural
network pre-trained on generic images, and adapts it to large-scale DR datasets. From images in the training set, we also automatically segment the abnormal patches with an occlusion test,and model the transformations and deterioration process of DR. Our results can be widely used for fast diagnosis of DR, medical education and public-level healthcare propagation.
DEEP FACIAL DIAGNOSIS: DEEP TRANSFER LEARNING FROM FACE RECOGNITION TO FACIAL...IRJET Journal
This document discusses using deep learning techniques like transfer learning and convolutional neural networks (CNNs) to perform computer-aided facial diagnosis of diseases from photographs. Specifically, it explores diagnosing single diseases like beta-thalassemia and multiple diseases including beta-thalassemia, hyperthyroidism, Down syndrome, and leprosy. The researchers achieve over 90% accuracy on these facial diagnosis tasks using deep transfer learning from pre-trained face recognition models, outperforming traditional machine learning methods. This shows promise for non-invasive disease screening using facial analysis with deep learning.
IRJET- Eye Diabetic Retinopathy by using Deep LearningIRJET Journal
This document discusses using deep learning techniques to detect diabetic retinopathy from retinal images. Diabetic retinopathy is a disease affecting the retina caused by complications of diabetes. It can lead to vision loss if not managed properly. The document proposes using methods like image preprocessing, feature extraction, segmentation, and classification with deep learning algorithms to automatically analyze retinal images and detect signs of diabetic retinopathy. Specifically, it involves extracting features from images using techniques like color density histograms, then training a deep learning model on those features to classify images according to the severity of diabetic retinopathy present. The goal is to enable early detection of the disease from retinal images to help prevent vision loss.
This document describes a proposed method for classifying chest x-ray images to diagnose lung infections using convolutional neural networks (CNNs). The objectives are to examine if transfer learning from different source domains can improve performance for classifying healthy, pneumonia and COVID-19 cases using a small dataset. The proposed methodology includes collecting datasets, training a CNN model using transfer learning, evaluating performance using a confusion matrix, and identifying opportunities for future enhancement like exploring different network architectures and domains.
Generalized deep learning model for Classification of Gastric, Colon and Rena...IRJET Journal
This document proposes developing a generalized deep learning model to classify gastric, colon, and renal cancer using a single model. The model would be trained on whole slide images of tissue samples fed through an EfficientNet model pre-trained on ImageNet. The model would be trained using transfer learning with partial transfusion to demonstrate the ability to classify pathology images from different sites. Previous studies have developed models to classify individual tissue types but not a unified model. The proposed model aims to address situations where the tissue site of origin is unknown.
Brain Tumor Detection From MRI Image Using Deep LearningIRJET Journal
This document presents a study on using deep learning techniques for brain tumor detection from MRI images. It proposes two Convolutional Neural Network models - one without transfer learning that achieves 81.42% accuracy, and one with transfer learning using the VGG16 architecture that achieves significantly higher accuracy of 98.8%. The study uses a dataset of over 5,000 MRI images categorized as normal, benign tumor, or malignant tumor. Data preprocessing techniques like filtering and enhancement are applied before training the models. Transfer learning helps reduce training time and improves model performance for tumor classification compared to training from scratch without transferring learned features.
This document describes a study that used deep learning models to diagnose Alzheimer's disease using MRI scans. Specifically, it used convolutional neural networks (CNNs) trained on MRI data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset. The study achieved an F1-score of 89% for classifying Alzheimer's vs normal cognition. It also used a CycleGAN for data augmentation, which generated additional synthetic MRI images and improved the CNN classification accuracy to 95% F1-score. The document provides background on Alzheimer's diagnosis, CNNs, GANs including CycleGAN, related work applying machine learning to Alzheimer's diagnosis, and the methodology used in this study for data preprocessing, model training, and evaluating the impact of GAN
APPLICATION OF CNN MODEL ON MEDICAL IMAGEIRJET Journal
The document discusses using convolutional neural network (CNN) models to detect diseases from medical images such as chest X-rays. It describes how CNN models can be trained on large labeled datasets of chest X-rays to learn patterns and features that indicate diseases. The document then evaluates several CNN architectures - including VGG-16, ResNet, DenseNet, and InceptionNet - for classifying chest X-rays as normal or infected. It finds these models achieve high accuracy, with metrics like accuracy over 89% and AUC over 0.94. In conclusion, deep learning models show promising results for automated disease detection from medical images.
Similar to Application of generative adversarial networks (GAN) for ophthalmology image domains - a survey.pdf (20)
Robustness in Machine Learning Explanations Does It Matter-1.pdfDaniel983829
This document discusses the desirability of robustness in machine learning explanations. It argues that robustness is desirable to the extent we want explanations to reflect real patterns in the data and the world. The importance of explanations capturing real patterns depends on the problem context. In some contexts, non-robust explanations can pose moral hazards. Clarifying how much we want explanations to capture real patterns can help determine whether the "Rashomon effect" is beneficial or problematic.
Robustness and Regularization of Support Vector Machines.pdfDaniel983829
This document summarizes a research paper that establishes an equivalence between robust optimization and regularization as applied to support vector machines (SVMs). It shows that regularized SVMs can be formulated as a robust optimization problem that minimizes error under potential adversarial noise or disturbances in the training data. This provides an alternative explanation for why regularization improves generalization performance of SVMs. The document also discusses implications for algorithm development and analysis, including a new proof of consistency for SVMs based on their robustness interpretation.
Robustness of Machine Learning Models Beyond Adversarial Attacks.pdfDaniel983829
This document discusses assessing the robustness of machine learning models to "real-world perturbations" that occur naturally rather than being crafted by an adversary. It argues that adversarial robustness is not a valid measure of robustness to such natural perturbations. The document proposes a probabilistic approach to compute the likelihood that a real-world perturbation would change a model's prediction, quantifying its real-world robustness. This approach models possible input perturbations individually for each application and works for any "black-box" model. It illustrates the approach on two datasets and analytically solvable cases, and discusses estimating real-world robustness in high-dimensional spaces.
Robustness in Deep Learning - Single Image Denoising using Untrained Networks...Daniel983829
This document is a thesis submitted by Esha Singh to the University of Minnesota for the degree of Master of Science in May 2021. The thesis explores single image denoising using untrained neural networks. It first provides background on deep learning, inverse problems, image denoising and neural networks. It then reviews existing image denoising algorithms including spatial/transform domain and neural network methods. The thesis also discusses recent work on deep image priors and rethinking single image denoising using over-parameterization and low-rank matrix recovery. Preliminary experiments on denoising images with salt and pepper noise are presented to demonstrate the proposed methodology.
Data preparation for artificial intelligence in medical imaging - A comprehen...Daniel983829
This document provides a comprehensive guide to tools and platforms for preparing medical image data for artificial intelligence applications. It discusses key steps in a medical image preparation pipeline including image acquisition, de-identification, data curation, storage, and annotation. A variety of open-access tools are reviewed that can perform image de-identification, data curation, storage, and annotation. Examples of medical imaging datasets covering different organs and diseases are also provided. The guide aims to enable standardized and large-scale data preparation and AI development in medical imaging.
Generative Adversarial Networks for Robust Medical Image Analysis.pdfDaniel983829
This document presents two approaches for improving robustness in medical image segmentation using generative adversarial networks (GANs). The first approach, UltraGAN, uses a GAN to enhance the quality of ultrasound images and improve robustness to low image quality. The second approach, MedRobGAN, generates adversarial medical image examples to improve robustness against adversarial attacks. Both methods are evaluated on medical segmentation tasks to validate their effectiveness in improving robustness.
Auditing AI models for verified deployment under semantic specifications.pdfDaniel983829
The document proposes a framework called AuditAI for auditing deep learning models prior to deployment. It does this by creating a bridge between the deep learning model and a generative model, allowing them to share the same semantically-aligned latent space. This enables the use of unit tests to verify that predefined specifications are satisfied for controlled variations in the latent space, like accuracy over 95% for different angles of faces. The framework applies interval bound propagation to provide certifications for specifications involving latent space perturbations, providing interpretable and scalable auditing of deep learning models.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
20 Comprehensive Checklist of Designing and Developing a WebsitePixlogix Infotech
Dive into the world of Website Designing and Developing with Pixlogix! Looking to create a stunning online presence? Look no further! Our comprehensive checklist covers everything you need to know to craft a website that stands out. From user-friendly design to seamless functionality, we've got you covered. Don't miss out on this invaluable resource! Check out our checklist now at Pixlogix and start your journey towards a captivating online presence today.
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
GraphSummit Singapore | The Art of the Possible with Graph - Q2 2024Neo4j
Neha Bajwa, Vice President of Product Marketing, Neo4j
Join us as we explore breakthrough innovations enabled by interconnected data and AI. Discover firsthand how organizations use relationships in data to uncover contextual insights and solve our most pressing challenges – from optimizing supply chains, detecting fraud, and improving customer experiences to accelerating drug discoveries.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
2. Page 2 of 19
You et al. Eye and Vision (2022) 9:6
provides an opportunity for early detection of diabetic
retinopathy, age-related macular degeneration, and glau-
coma [2]. However, there are still many constraints on the
use of data-driven artificial intelligence (AI) models in
ophthalmology. Ocular imaging data have been expand-
ing globally [3], but there is a shortage of high-quality
images and pathological data from patients to train AI
models [4]. In addition, there are many diagnostic ocu-
lar imaging modalities such as color fundus photography,
retinal OCT, ultra-widefield fundus photography, retinal
angiography, ultrasonography, corneal tomography, and
anterior segment OCT. Because imaging techniques dif-
fer from each other in terms of structural complexity and
image dimensionality, the need for cross-modality image
processing methods has increased to improve the disease
prediction models.
In recent years, generative adversarial network (GAN)
has become the technique of choice for image genera-
tion and translation in the field of medical imaging [5].
GAN, which is a new type of deep learning developed by
Ian Goodfellow [6], can automatically synthesize medi-
cal images by learning the mapping function from an
arbitrary distribution to the observed data distribution,
which is the process of extracting mathematical relation-
ships from data distributions for matching input to out-
put data. As deep learning requires more data to build
better accurate models, the medical research community
requires more databases from various imaging modalities
such as computed tomography (CT) and magnetic reso-
nance imaging (MRI) [7]. Accordingly, many experiments
have been performed to demonstrate the benefit of using
GAN, which can generate realistic synthetic images. Pre-
vious publications in radiology have shown that the appli-
cation of GAN includes data augmentation, denoising,
super-resolution, domain transfer between modalities,
and segmentation [8]. They demonstrated that the poten-
tial gains from GAN can improve deep learning models
for pathological conditions and can support clinicians for
diagnosis using complex medical images. While medical
image processing using GAN has been actively studied in
radiology [9], there have been relatively few studies using
GAN in the field of ophthalmology image domains.
A previous literature review showed that the adop-
tion of GAN for medical imaging is increasing rapidly
[10], but it is not familiar to researchers in the field of
ophthalmology. Since ophthalmology imaging tech-
niques are becoming more important for diagnosing
ocular diseases, the use of GAN will gradually increase
to achieve a more accurate diagnosis. In this work, we
present a literature review on the application of GAN
in ophthalmology image domains to discuss important
contributions and identify potential future research
directions. This paper attempts to guide future research
on image processing in the ophthalmic domain through
the proper application of GAN techniques.
Review
Overview
We detail the studies using GAN for ophthalmology
image domains from available literature. We sum-
marized how GAN was utilized in the field of oph-
thalmology imaging. The type of GAN used in the
analysis, task, imaging domain, and outcome of the
application of GAN were collected to verify its useful-
ness. We searched for potentially relevant literature in
PubMed, Embase, and Google Scholar databases using
the following search strategy: ((generative adversar-
ial network) OR (GAN) OR (deep generative model))
AND ((ophthalmology) OR (diabetic retinopathy) OR
(age-related macular degeneration) OR (fundus pho-
tography) OR (optical coherence tomography)). The
initial selection of studies was performed based on
texts with titles and abstracts. We included only peer-
reviewed articles published before June 2021. Arti-
cles that did not contain original research using GAN
were excluded. Only articles written in English were
included in the study, and studies without a specific
description of the GAN model were excluded. In the
case of multiple publications of the same research, we
regarded them as one study. The initial search yielded
855 articles. Of these, 806 were removed because
their studies or manuscripts were not related to either
GAN or ophthalmology images. A further three were
excluded because they were duplicates of other stud-
ies in the list or were not research articles. Finally, 48
articles were included in the final literature review. To
limit bias in the search, additional searches were per-
formed using other common ocular diseases (such as
dry eye, conjunctivitis, cataract, glaucoma, retinal vein
occlusion, central serous chorioretinopathy, and stra-
bismus) and other imaging modalities found in the
previous search. However, no additional research arti-
cles were obtained.
Since a comprehensive list of ophthalmology image
datasets has been provided in previous studies [3, 11],
we found no reason to discuss ocular image datasets in
this work. Traditional data augmentation refers to an
increase in the number of training examples through
the rotation, flipping, cropping, translation, and scaling
of existing images to improve the performance of deep
learning models, and can also be used to train GAN
models. Generally, most GAN techniques rely on large
amounts of annotated data, although several studies
have targeted data augmentation in a small amount of
pathological data.
3. Page 3 of 19
You et al. Eye and Vision (2022) 9:6
GAN techniques
This section provides the general concepts of GAN for
analyzing ophthalmology images, especially the archi-
tectures most frequently encountered in the literature
reviewed. The basic structure of a GAN is called a vanilla
GAN [6]. The architecture of the vanilla GAN consists of
two separate deep learning models, including the genera-
tor, which synthesizes candidate samples based on the
data distribution of the original dataset, and a discrimina-
tor, which tries to distinguish the synthesized candidate
samples from the real samples from the original dataset.
These two modules are trained simultaneously because
the gradient information is back-propagated to the gen-
erator to increase realistic image synthesis capabilities
and to the discriminator to increase real/fake discrimi-
nating capabilities. After vanilla GAN was introduced,
GAN was highlighted because of its ability to generate
realistic synthetic images based on the original dataset.
Figure 1 illustrates the structure of the vanilla GAN and
retinal images generated. In the example of retinal image
synthesis, vectors of late space are randomly selected ini-
tially; however, after training, they obtain an appropriate
functional relationship with the generated image. Ran-
domly generated images and original real retinal images
are classified by the discriminator and this result is back-
propagated and reflected in the training of both the gen-
erator and the discriminator. Finally, the desired outcome
after training the GAN is that the pixel distributions from
the generated retinal images should approximate the
distribution of real original retinal images. According to
the original paper describing GAN, the generator is like
a team that produces counterfeit money, which wants
to use counterfeit money without it being detected, and
the discriminator is like the police detecting counterfeit
money. The competition between the two teams results
in better counterfeiting [6]. After image generation using
GAN was popularized, novel GAN techniques were con-
stantly developed in the machine learning community.
In this review, we found that most studies on ophthal-
mology images have used progressively growing GAN
(PGGAN), conditional GAN, Pix2pix, and cycle-consist-
ent GAN (CycleGAN).
Currently, vanilla GAN is not widely used because
of its low-quality outputs and instability during train-
ing. However, it has been the basis of recent GAN vari-
ant techniques (Table 1). A deep convolutional GAN
(DCGAN) is based on the vanilla GAN by replac-
ing the building block with fully convolutional layers
[10]. Wasserstein GAN is an improved version of the
vanilla GAN that uses a metric of the distance between
two probability distributions (Wasserstein distance)
as a loss function [12]. PGGAN is an extension of the
vanilla GAN with a progressively growing generator
and discriminator to generate realistic high-resolution
images. The main concept of PGGAN is to build gener-
ators and discriminators, starting from a low-resolution
to a high-resolution network. The newly added layers
model fine-grained details as the training progresses.
As the images are generated from a random noise vec-
tor, the original PGGAN cannot generate new instances
with objects in the desired condition. Additionally,
StyleGAN is a variant of PGGAN that adds the style
transfer function in a conditional setting to the archi-
tecture of PGGAN [13]. Style and feature changes in
Fig. 1 An illustration of a basic architecture of GAN (vanilla GAN) for retinal image synthesis. The generator transforms a noise vector z from the
distribution p(z)into a synthesized retinal image xg. The discriminator distinguishes the synthetic and real retinal images based on the distributions
of xg and xr, respectively. The generated image samples form a distribution pg(x), which is desired to be an approximation of pr(x)from real image
sample, after successful training
4. Page 4 of 19
You et al. Eye and Vision (2022) 9:6
Table
1
The
characteristics
of
typical
GAN
variant
techniques
and
examples
of
general
tasks
in
general
medicine
and
ophthalmology
fields
CT
=
computed
tomography;
GAN
=
generative
adversarial
network;
MRI
=
magnetic
resonance
imaging;
OCT
=
optical
coherence
tomography;
PET
=
positron
emission
tomography
GAN
Techniques
Dataset
Characteristics
Task
examples
in
general
medicine
Task
examples
in
ophthalmology
Deep
convolutional
GAN
(DCGAN)
Images
from
one
domain
Improved
image
quality
using
deep
convolutional
layers
Augmentation
of
CT
images
[88]
Fundus
photographs
synthesis
[39]
Wasserstein
GAN
(WGAN)
Images
from
one
domain
Using
Wasserstein
distance
as
a
loss
function
Augmentation
of
CT
images
[89]
Removing
artifacts
in
CT
[90]
Anomaly
detection
[25]
OCT
segmentation
[91]
Progressively
growing
GAN
(PGGAN)
Images
from
one
domain
(generally
high
resolution)
High
resolution
&
realistic
image
generation
X-ray
image
synthesis
[92]
Data
augmentation
for
cytological
images
[93]
Data
augmentation
for
fundus
photog-
raphy,
retinal
OCT,
and
ocular
images
[40,
46,
82]
Super-resolution
of
fundus
photo-
graphs
[55]
StyleGAN
Images
of
one
domain
or
multiple
domains
(unpaired
images)
Disentanglement
of
representations
(mapping
features
to
low
dimensions)
Augmentation
of
CT
and
MRI
images
in
specific
conditions
[13,
94]
Skin
image
synthesis
[95]
None
Conditional
GAN
(vector
input
models)
Images
annotated
by
conditional
variables
Image
synthesis
conditioned
to
specific
variables
Super-resolution
guided
by
a
condi-
tional
variable
[96]
Data
augmentation
for
retinal
OCT
[44]
Post-intervention
(orbital
decompres-
sion)
prediction
[15]
Conditional
GAN
(Pix2pix
and
other
image
input
models)
Paired
images
of
two
domains
or
classes.
(Training
samples
should
be
aligned)
Supervised
learning
for
image-to-
image
translation
Super-resolution
for
fluorescence
microscopy
images
[97]
Domain
transfer
(CT
→
PET)
[98]
Segmentation
of
lungs
from
chest
X-ray
[99]
CT
image
synthesis
[100]
Domain
transfer
(fundus
photogra-
phy
→
angiography)
[62]
Retinal
OCT
segmentation
[36]
Retinal
vessel
segmentation
[28]
Data
augmentation
for
fundus
photog-
raphy
and
corneal
topography
[17,
47]
Super-resolution
GAN
(SRGAN)
Low-
and
high-resolution
image
pairs
Adopting
perceptual
loss
to
generate
super-resolved
realistic
images
Super-resolution
for
dental
X-ray
[101]
Super-resolution
for
optic
disc
photog-
raphy
[56]
Cycle-consistent
GAN
(CycleGAN)
Unpaired
images
of
two
domains
or
classes
Adopting
a
cycle
consistency
for
domain
transfer
without
any
paired
dataset
Manipulating
breast
imaging
[102]
Data
augmentation
for
CT
and
throat
images
[103,
104]
Segmentation
for
cardiac
ultrasound
[105]
Denoising
for
fundus
photography
and
OCT
[22,
53]
Domain
transfer
(Ultra-widefield
retinal
images
→
classic
fundus
photography)
[63]
StarGAN
Unpaired
images
of
multiple
domains
or
classes
A
single
network
to
achieve
transla-
tion
of
multiple
domains
Domain
transfer
between
MRI
con-
trasts
[24]
None
5. Page 5 of 19
You et al. Eye and Vision (2022) 9:6
synthetic images can be performed using an additional
mapping network for the latent space in the generator
of StyleGAN.
In many cases, synthetic images should be generated
with the desired properties to adopt GAN for medi-
cal purposes. A conditional GAN is an extended archi-
tecture of vanilla GAN, where both the generator and
discriminator are trained using not only the original
dataset but also additional conditioning variables [14].
To achieve good image generation performance in multi-
ple domains, researchers have modified the generators of
conditional GAN in various deep learning architectures.
Currently, conditional GAN includes many types of GAN
models because the condition variable can be any vari-
able including a single status variable [15], images of the
same or different domains [16], masked images [17], and
guided heatmap images [18]. If the conditional variable
is set as an image, the training dataset should commonly
contain aligned image pairs. The most widely used form
of conditional GAN is Pix2pix, which contains an image-
to-image translation framework [19]. Instead of using a
conventional encoder-decoder as a generator, Pix2pix
adopts a U-Net-like architecture with skip connections to
generate synthetic images from the input images. In Pix-
2pix, the discriminator is used at the local image patch
level to improve the performance. The GAN architecture
for a super-resolution task (SRGAN) was developed by
adopting a conditional GAN and perceptual loss [20].
However, conditional GAN models, including Pix2pix,
have a critical disadvantage in that the shortage of paired
datasets restricts their application to real problems.
Recently, CycleGAN was developed to generate images
without matching paired images [21]. CycleGAN involves
the simultaneous training of two generators and two dis-
criminators. The CycleGAN adopts a cycle consistency,
which is based on the idea that the output of the first
generator can be used as input to the second generator,
and the output of the second generator should be like the
original image. This cycle consistency allows CycleGAN
to learn the characteristics of the two image domains
to transfer the domains without any paired dataset. The
weights for the training parameters of CycleGAN mod-
ules can be tuned depending on the image domain or
task. CycleGAN can perform denoising by mapping clean
and noisy domains from unpaired training data [22]. Cur-
rently, variants of CycleGAN, such as StarGAN [23] and
its variants [24], have been introduced to achieve high
performance in a multiple domain transfer problem. The
characteristics of typical GAN techniques and examples
of general tasks in general medicine (especially radiology)
and ophthalmology fields are summarized in Table 1.
It should be noted that there are several cases where it
is not classified as a specific type of GAN because the
custom architectures of GAN were commonly designed
for each imaging domain.
Applications in ophthalmology
Here, we survey the literature on GAN for ophthalmology
image domains. Applications are introduced according to
the type of tasks of GAN models, including segmentation
(15 studies), data augmentation (11 studies), denoising (8
studies), domain transfer (8 studies), super-resolution (4
studies, two studies overlap with denoising), post-inter-
vention prediction (3 studies), and feature extraction (2
studies). Figure 2 shows examples of the applications of
GAN. Figure 3 shows the number of studies based on the
tasks of the GAN and image domains. Some studies that
handled the two image domains were double-counted.
Most studies have focused on the generative aspect of
GAN, and only two studies, f-AnoGAN [25] and AMD-
GAN [26] adopted the discriminative aspect with feature
extraction. The survey showed that segmentation was
the most studied task in GAN in ophthalmology. Among
ophthalmology imaging domains, fundus photography
(24 studies) has been most frequently analyzed using
GAN in the literature. GAN has also been used in vari-
ous imaging domains, including retinal OCT (15 studies),
retinal angiography (7 studies), ultra-widefield fundus
photography (scanning laser ophthalmoscopy, 3 studies),
anterior segment OCT (two studies), periorbital facial
image for orbital diseases (1 study), ocular surface image
(1 study), corneal topography (1 study), meibography
infrared imaging (1 study), and in vivo corneal confocal
microscopy (1 study). If one study deals with two modali-
ties, it was reviewed and double-counted if necessary.
Although conditional GAN is most frequently mentioned
in the survey, it is difficult to conclude that it has been
most widely used because the conditional GAN mod-
els refer to a wide variety of deep learning structures for
each study. A table detailing the literature is provided for
each section of the application.
Segmentation
Image segmentation is a task where pixels or areas in
an image are assigned a category label. Segmentation is
the most frequently studied (14 studies) focusing on the
identification of structures such as retinal vessels, retinal
layers, and optic nerve. Identifying pathological areas on
the ocular images can help clinicians to diagnose more
accurately, and thus segmentation is an important task
for developing AI models for medicine. Table 2 shows a
summary of the literature review for the segmentation
task using GAN.
GAN techniques are typically used to segment reti-
nal vessels from fundus photographs. For decades, reti-
nal vessel segmentation has been a challenging problem
6. Page 6 of 19
You et al. Eye and Vision (2022) 9:6
in the computer science community because vessels
have various widths, colors, tortuosity, and branching.
After conditional GAN was applied to this problem [27],
many variants of the conditional GAN were proposed by
modifying the architectures. In particular, Son et al. [28]
improved the conditional GAN using a generator based
on U-Net, similar to the Pix2pix architecture. To improve
segmentation performance, some studies employed
patch-based GAN [29], multi-kernel pooling layers [30],
topological structure-constrained models [31], large
Fig. 2 Examples of applications of GAN in ophthalmology image domains. a Post-intervention prediction for decompression surgery for thyroid
ophthalmopathy [15] and anti-vascular endothelial growth factor (VEGF) therapy for neovascular age-related macular degeneration [66]. b
Denoising in fundus photography [53] and peripapillary optical coherence tomography (OCT) [16]. c Super-resolution for optic nerve head
photography [56]. d Domain transfer for fundus photography to angiography [62] and ultra-widefield to classic fundus photography (re-analysis
in this work) [63]. e Data augmentation for ocular surface images [46] and anterior segment OCT [82]. f Segmentation for corneal sub basal nerves
in in vivo confocal microscopy images [37]. Most images were generated according to publicly available datasets and the methods of each
study (some cases are based on our own dataset)
Fig. 3 Number of studies that were reviewed in this work grouped according to tasks and image domains. a Study objectives in the application of
GAN. b Ophthalmology image domains for the use of GAN. If one study deals with two issues, it was reviewed and double-counted appropriately
7. Page 7 of 19
You et al. Eye and Vision (2022) 9:6
Table
2
Summary
of
literature
review
for
image
segmentation
task
using
GAN
in
ophthalmology
imaging
domains
GAN
=
generative
adversarial
network;
OCT
=
optical
coherence
tomography;
WGAN
=
Wasserstein
GAN
Publication
Basic
technique
Domain
Target
Summary
Iqbal
et
al.
[27]
Conditional
GAN
Fundus
photography
Retinal
vessels
The
framework
achieved
retinal
vessels
image
segmentation
from
a
small
training
set
Wang
et
al.
[33]
Conditional
GAN
(using
PatchGAN)
Fundus
photography
Optic
disc
and
optic
cup
Unsupervised
domain
adaptation
for
joint
optic
disc
and
cup
segmentation
using
a
patch-based
discriminator
Son
et
al.
[28]
Conditional
GAN
(using
U-Net
as
a
generator)
Fundus
photography
Retinal
vessels
The
GAN
model
segmented
retinal
vasculature
and
the
optic
disc
using
a
dataset,
which
consisted
of
fundoscopic
images
and
manual
segmentation
of
the
vessels
Rammy
et
al.
[29]
Conditional
GAN
Fundus
photography
Retinal
vessels
Patch-based
GAN
with
additional
loss
function
to
learn
thin
and
thick
vessel
segmented
retinal
vessels
with
the
enhanced
performance
Park
et
al.
[30]
Conditional
GAN
Fundus
photography
Retinal
vessels
The
proposed
GAN
model
with
a
multi-kernel
pooling
and
false
negative
loss
could
segment
retinal
blood
vessels
more
robustly
Heisler
et
al.
[36]
Pix2Pix
(conditional
GAN)
Peripapillary
retinal
OCT
Nerve
fiber
layer,
Bruch’s
mem-
brane,
choroid-sclera
boundary
The
use
of
a
generative
adversarial
network
and
unlabeled
data
can
improve
the
performance
of
segmentation
for
the
3D
morphometric
analysis
of
glaucomatous
optic
nerve
head
volumes
Yang
et
al.
[31]
Conditional
GAN
(topological
structure-constrained)
Fundus
photography
Retinal
vessels
The
topological
structure-constrained
proposed
GAN
model
identified
retinal
arteries
and
veins
via
segmentation
from
the
complex
background
of
retinal
images
Yang
et
al.
[106]
Conditional
GAN
Fundus
photography
Retinal
vessels
To
separate
blood
vessels
from
fundus
image,
the
model
could
detect
more
tiny
vessels
and
locate
the
edge
of
blood
vessels
more
accurately
Zhao
et
al.
[32]
Conditional
GAN
(with
a
large
receptive
field)
Fundus
photography
Retinal
vessels
The
proposed
retinal
vessel
segmentation
algorithm
using
GAN
with
a
large
receptive
field
could
capture
large-scale
high-level
semantic
vessel
features
Kadambi
et
al.
[34]
Wasserstein
GAN
Fundus
photography
Optic
disc
and
optic
cup
WGAN-based
domain
adaptation
showed
a
better
perfor-
mance
than
baseline
models
for
the
joint
optic
disc-and-cup
segmentation
in
fundus
images
Bian
et
al.
[35]
Conditional
GAN
(using
U-Net
as
a
generator)
Fundus
photography
Optic
disc
and
optic
cup
The
proposed
method
successfully
performed
optic
disc
and
cup
segmentation.
The
cup-to-disc
ratio
was
automatically
calculated
with
a
good
performance
Khan
et
al.
[38]
Conditional
GAN
Meibography
infrared
images
Meibomian
gland
The
proposed
GAN
automatically
identified
the
area
of
the
meibomian
glands
and
outperformed
the
state-of-art
methods
Yildiz
et
al.
[37]
Conditional
GAN
In
vivo
corneal
confocal
micros-
copy
images
Sub-basal
nerves
Automatic
segmentation
of
sub-basal
nerves
in
in
vivo
confocal
microscopy
images
was
performed
using
the
U-Net
and
GAN-based
techniques
as
a
diagnostic
tool
for
corneal
diseases
Zhou
et
al.
[107]
Conditional
GAN
(using
U-Net
as
a
baseline
architecture)
Fundus
photography
Retinal
vessels
The
GAN
model
strengthened
retinal
vessel
segmentation
in
the
low-contrast
background
using
a
symmetric
equilibrium
GAN
(U-Net-based),
multi-scale
features
refine
blocks
and
attention
mechanism
8. Page 8 of 19
You et al. Eye and Vision (2022) 9:6
receptive fields [32], and symmetric equilibrium genera-
tors with attention mechanisms [17]. Since most of these
studies have been conducted using limited annotated
vessel datasets without collaboration with ophthalmolo-
gists, validation with real patient data was not performed.
In addition, accurate cup-to-disc ratio calculation
based on optic disc segmentation is an important prob-
lem for evaluating optic nerve damage and glaucoma.
Damage to the optic nerve increases the cup-to-disc
ratio, which is difficult to compare with the naked eye if
the damage is small. In recent studies, GAN was useful
for segmenting the optic disc and cup in fundus photo-
graphs using patch-based conditional GAN [33] and
Wasserstein GAN [34]. The cup-to-disc ratio, a clini-
cally used measurement to assess glaucoma progression,
can be directly calculated from the optic cup and disc
segmentation using conditional GAN [35]. This study
showed comparable performance in assessing the cup-to-
disc ratio for glaucoma screening.
Another application found in the literature is the seg-
mentation of retinal layers in OCT images. Retinal lay-
ers consist of the vasculature, neurons, glia, and their
connections, and each layer changes differently under
pathological conditions. Pix2pix was successfully applied
to segment the retinal nerve fiber layer, Bruch’s mem-
brane, and choroid-sclera boundary in peripapillary reti-
nal OCT images [36]. GAN was also applied to evaluate
corneal pathological conditions using in vivo confocal
microscopy images [37]. In this study, the segmentation
of corneal sub basal nerves was achieved using a condi-
tional GAN to detect corneal diseases. The meibomian
gland can be evaluated by the GAN to segment the area
of the meibomian glands in meibography infrared images
[38]. In this study, the conditional GAN outperformed
U-Net and masked regions with convolutional neural
networks (mask R-CNN).
Data augmentation
The development of a machine-learning model requires
enough data. Imbalanced data is a barrier to the training
model, and the lack of data often presents in many medi-
cal problems because of barriers to access and usability
[3]. Traditional data augmentation is commonly unable
to extrapolate the generated data, which leads to data bias
and suboptimal performance of trained models. Many
researchers have shown that data augmentation using
GAN techniques can provide additional benefit over
traditional methods [8]. Recently, GAN techniques have
been widely used to synthesize realistic medical images
for data augmentation. Here, 11 studies investigated data
augmentation for ophthalmology imaging domains (see
Table 3). Several studies using GAN have focused on fun-
dus photography and retinal OCT image generation to
augment training datasets for machine learning.
In recent years, generating realistic fundus photo-
graphs has become a challenging issue. DCGAN was
initially used to generate synthetic peripapillary fundus
photographs [39]. The machine learning model based on
DCGAN showed better diagnostic performance for glau-
coma detection than conventional deep learning models.
Burlina et al. evaluated the performance of a PGGAN to
generate realistic retinal images [40]. In their study, two
retinal specialists could not distinguish real images from
synthetic images. Zhou et al. used a conditional GAN
to generate high-resolution fundus photographs based
on structural and lesion mask images [17]. The multi-
channel generator technique, which trains multiple GAN
models for each feature such as exudates, microaneu-
rysms, and bleeding, was used to augment fundus pho-
tographs to overcome the imbalance of data to build a
diabetic retinopathy detection model [41].
The generation of synthetic retinal OCT is another
important task for data augmentation in the development
of machine-learning models for automated OCT diag-
nosis. By integrating both normal and pathological data,
GAN can generate synthetic OCT images with various
pathological grades for data augmentation [42]. Zheng
et al. showed that realistic retinal OCT images could be
generated using PGGAN, which could improve the clas-
sification performance of deep learning models [43]. Data
augmentation based on conditional GAN also improved
the segmentation performance of retinal OCT images
[44]. CycleGAN was applied to OCT data augmentation
for rare retinal diseases in a few-shot learning system
design [45]. GAN has been used for data augmentation of
anterior OCT images for angle-closure glaucoma, ocular
surface images for conjunctival disease [46] and corneal
topography images for keratoconus detection [47].
Denoising & super‑resolution
Image enhancement tasks such as denoising and super-
resolution are important because ophthalmology images
generally suffer from limitations of the device, the skill
of the examiner, variations of ocular anatomy, and trans-
parency of the visual axis. Image quality may affect the
diagnostic performance using ocular images although
the device and software offer suppression of noise and
artifacts. As each imaging domain has characteristic
noise and artifacts, several research groups have tried
to develop data-driven GAN models tailored to each
domain. There are 10 studies investigating denoising or
super-resolution for ophthalmology imaging domains
(detailed in Table 4).
9. Page 9 of 19
You et al. Eye and Vision (2022) 9:6
To remove speckle noise in retinal OCT images, a
conditional GAN model with Wasserstein distance and
perceptual loss was proposed [48]. Huang et al. showed
that both super-resolution and noise reduction can be
performed simultaneously using a conditional GAN
[49]. Cheong et al. built DeShadowGAN using manually
masked artifact images and conditional GAN with per-
ceptual loss and demonstrated the effectiveness of the
model in removing shadow artifacts [16]. Similarly, con-
ditional GAN has also been applied to remove speckle
noise in peripapillary retinal OCT [50] and anterior
segment OCT [51]. However, image denoising methods
using conditional GAN can match low- and high-quality
image pairs; however, these data are typically unavailable
in the medical field. Therefore, Das et al. used CycleGAN
for super-resolution and noise reduction in retinal OCT
images to facilitate unpaired image datasets [52].
Fundus photography has several artifacts and noise,
including overall haze, edge haze, arcs, and lashes. In
a super-resolution problem, artificial manipulation to
reduce image resolution is possible. Since it is difficult
to collect paired clean and artifact fundus photographs,
CycleGAN has been used to improve the image quality
of fundus photography [53, 54]. These studies showed
that CycleGAN can effectively reduce artifacts to pro-
vide clearer retinal images to clinicians. PGGAN was
also shown with a conditional design that was applied
to super-resolution fundus photography [55]. Ha et al.
adopted that to generate high-resolution synthetic optic
disc images with a 4-times up-scaling using SRGAN [56].
Domain transfer
Most machine learning works have performed the devel-
opment and validation of data from the same domain. To
build a more generalized machine-learning model, data
from different domains might be fused through domain
transfer, which is the transfer between different imag-
ing modalities. The domain transfer task of GAN is the
cross-modality image synthesis process by which images
are generated for one modality based on another. Cross-
domain modality using the domain transfer technique
has shown the possibility of obtaining additional clinical
information without additional examinations [57]. Eight
studies using GAN mainly focused on domain transfer
for ophthalmology imaging domains (shown in Table 5).
Notably, several studies that used image transfer genera-
tors were categorized as data augmentation tasks because
they focused on image synthesis tasks. The concept of
conditional GAN has been used in most studies because
the generator must have an image input channel as a con-
ditional variable for domain transfer. Generally, GAN
models without conditional inputs can generate new
images infinitely by adjusting the latent vector, whereas
one input corresponds to one output image in typical
conditional GAN. GAN techniques allow for more high-
dimensional image transformation and realistic outputs
than simple pixel-level transformation.
Initially, Costa et al. demonstrated that a conditional
GAN can be used to generate realistic fundus photo-
graphs guided by masked vessel network images [58];
an autoencoder was used to synthesize new retinal ves-
sel images apart from training the GAN. Zhao et al. also
built a conditional GAN model that emphasizes the
ability to learn with a small dataset [59]. Extending the
conditional GAN, modified Pix2pix synthesized realis-
tic color fundus photographs to enlarge the image data-
set based on multiple inputs of the vessel and optic disc
masked images [60]. Wu et al. showed that retinal auto-
fluorescence images could be synthesized based on reti-
nal OCT data using a conditional GAN framework [61].
In that study, en-face OCT images from volumetric OCT
data were successfully transformed into synthetic auto-
fluorescence images to detect geographic atrophy regions
in the retina. Tavakkoli et al. demonstrated that realis-
tic retinal angiography images with diabetic retinopathy
were generated via conditional GAN using fundus pho-
tographs [62]. Although the model was trained using a
limited angiography dataset without detailed phase infor-
mation, the study showed the potential of image domain
transfer for the diagnosis of diabetic retinopathy. Based
on CycleGAN, ultra-widefield fundus photography can
be transformed into classic color fundus photography
to integrate retinal imaging domains [63]. In contrast,
a study converting classic fundus photography to ultra-
widefield fundus photography via CycleGAN was also
reported [64]. Larzaridis et al. demonstrated that time-
domain OCT could be converted to spectral-domain
OCT using conditional GAN with Wasserstein distance
and perceptual loss, showing that the integrated dataset
fused by the GAN improved the statistical power of the
OCT measurements.
Post‑intervention prediction
The aim of post-intervention prediction is to gener-
ate an image that explains how the anatomical appear-
ance changes after treatment. Three studies investigated
post-intervention prediction tasks for ophthalmology
imaging domains (detailed in Table 6). As post-interven-
tion results are represented as images in several medi-
cal fields, this task is useful to clinicians and patients to
understand how the intervention will affect the progno-
sis of diseases. However, the included studies have sev-
eral limitations in terms of short-term follow-up periods
for prediction and unstandardized interventions [65].
In addition, attention should be paid to interpreting the
results because anatomical prediction after treatment is
10. Page 10 of 19
You et al. Eye and Vision (2022) 9:6
not necessarily related to functional outcomes such as
visual acuity.
Yoo et al. proposed a postoperative appearance pre-
diction model for orbital decompression surgery for
thyroid ophthalmopathy using a conditional GAN
[15]. Although the experiment was performed at a
relatively low resolution, the results show the poten-
tial of GAN as a decision support tool for oculoplastic
and cosmetic surgeries related to the orbit. Two stud-
ies demonstrated that conditional GAN models could
predict OCT images after anti-vascular endothelial
growth factor (anti-VEGF) injection based on pre-
injection OCT images with exudative age-related mac-
ular degeneration. Liu et al. showed that the Pix2pix
model could generate synthetic post-injection OCT
using pre-injection images to estimate the short-term
response [66]. Lee et al. designed a conditional GAN
with a multi-channel input for anti-VEGF injection
[67]. The model was trained using both pre- and post-
injection OCT images as well as fluorescein angiog-
raphy and indocyanine green angiography to predict
post-injection OCT.
Table 3 Summary of literature review for data augmentation task using GAN in ophthalmology imaging domains
GAN = generative adversarial network; DCGAN = deep convolutional GAN; OCT = optical coherence tomography; PGGAN = progressively growing GAN
Publication Basic technique Domain Summary
Diaz-Pinto et al. [39] DCGAN Peripapillary fundus photography (optic disc
photo)
DCGAN was able to generate high-quality synthetic
optic disc images
Burlina et al. [40] PGGAN Fundus photography The GAN technique was used to synthesize high-
resolution realistic fundus images serving as proxy
data sets for use by retinal specialists and deep
learning models
Zheng et al. [43] PGGAN Retinal OCT (spectral domain) The image quality of real images vs. synthetic OCT
images generated by GAN was similar; the synthetic
OCT images were able to serve as augmentation of
training datasets for deep learning models
Zhou et al. [17] Conditional GAN Fundus photography To generate a large amount of balanced training
data, the GAN model synthesized high-resolution
diabetic retinopathy fundus images which can
be manipulated with arbitrary grading and lesion
information
Wang et al. [41] Multi-channel
GAN (modified
vanilla GAN)
Fundus photography The model generated a series of sub-fundus images
corresponding to the scattering diabetic retinopa-
thy features and made full use of both labeled and
unlabeled data
He et al. [42] Label smoothing
GAN (modified
vanilla GAN)
Retinal OCT The GAN model generated the synthetic unlabeled
images from limited OCT training samples, and the
mixing of the synthetic images and real images can
be used as training data to improve the classification
performance
Yoo et al. [45] CycleGAN Retinal OCT GAN generated OCT images of rare diseases from
normal OCT images and increased the accuracy of
diagnosing rare retinal diseases with few-shot clas-
sification
Kugelman et al. [44] Conditional GAN Retinal OCT (patch level) GAN was feasible to generate patches that are visu-
ally indistinguishable from their real variants and
improved the segmentation performance
Zheng et al. [82] PGGAN Anterior Segment OCT The synthetic OCT images generated by GAN
appeared to be of good quality, according to the
glaucoma specialists, and the deep learning model
for angle-closure detection was improved using
both synthetic and real images
Yoo et al. [46] CycleGAN, PGGAN Ocular surface image To improve the diagnostic accuracy, GAN was
adopted to perform data augmentation of ocular
surface images with conjunctival melanoma
Abdelmotaal et al. [47] Pix2pix Corneal topography (Scheimpflug images) The synthesized images showed plausible subjec-
tively- and objectively-assessed quality. Training
deep learning with a combination of real and
synthesized images showed better classification
performance to detect keratoconus
11. Page 11 of 19
You et al. Eye and Vision (2022) 9:6
Table
4
Summary
of
literature
review
for
image
enhancement
(denoising
and
super-resolution)
tasks
using
GAN
in
ophthalmology
imaging
domains
GAN
=
generative
adversarial
network;
OCT
=
optical
coherence
tomography;
PGGAN
=
progressively
growing
GAN
Publication
Basic
technique
Domain
Target
Summary
Halupka
et
al.
[48]
Modified
Wasserstein
GAN
+
perceptual
loss
(conditional
GAN)
Retinal
OCT
(spectral
domain)
Removing
speckle
noise
The
GAN
was
used
to
reduce
speckle
artifacts
in
retinal
OCT
images.
The
method
improved
the
image
quality
metrics
for
OCT
Mahapatra
et
al.
[55]
PGGAN
with
a
conditional
design
Fundus
photography
Super-resolution
Image
super-resolution
using
multi-stage
PGGAN
outperforms
competing
methods
and
baseline
GANs.
The
super-resolved
images
can
be
used
for
landmark
and
pathol-
ogy
detection
Huang
et
al.
[49]
Conditional
GAN
Retinal
OCT
Super-resolution
and
removing
noise
The
GAN
model
effectively
suppressed
speckle
noise
and
super-resolved
OCT
images
at
different
scales
Ouyang
et
al.
[51]
Conditional
GAN
Anterior
Segment
OCT
Removing
speckle
noise
The
model
removed
undesired
specular
arti-
facts
and
speckle-noise
patterns
to
improve
the
visualization
of
corneal
and
limbal
OCT
images
Yoo
et
al.
[53]
CycleGAN
Fundus
photography
Removing
artifacts
and
noise
The
GAN
model
removed
the
artifacts
auto-
matically
in
a
fundus
photograph
without
matching
paired
images
Cheong
et
al.
[16]
DeshadowGAN
(modified
conditional
GAN
with
perceptual
loss)
Peripapillary
retinal
OCT
(spectral
domain)
Removing
vessel
shadow
artifacts
The
GAN
model
using
manually
masked
artifact
images
and
perceptual
loss
function
removed
blood
vessel
shadow
artifacts
from
OCT
images
of
the
optic
nerve
head
Chen
et
al.
[50]
Conditional
GAN
Peripapillary
retinal
OCT
(spectral
domain)
Removing
speckle
noise
The
GAN
model
was
designed
for
speckle
noise
reduction
in
OCT
images
and
preserved
the
textural
details
found
in
OCT
Das
et
al.
[52]
CycleGAN
Retinal
OCT
Super-resolution
and
removing
noise
To
achieve
denoising
and
super-resolution,
adversarial
learning
with
cycle
consistency
was
used
without
requiring
aligned
low–high
resolution
pairs
Ha
et
al.
[56]
Enhanced
super-resolution
GAN
(SRGAN)
Peripapillary
fundus
photography
(optic
disc
photo)
Super-resolution
The
GAN
approach
was
capable
of
4-times
up-scaling
and
enhancement
of
anatomical
details
using
contrast,
color,
and
brightness
improvement
Yuhao
et
al.
[54]
CycleGAN
Fundus
photography
Removing
artifacts
and
noise
The
developed
model
dehazed
cataractous
retinal
images
through
unpaired
clear
retinal
images
and
cataract
images
12. Page 12 of 19
You et al. Eye and Vision (2022) 9:6
Feature extraction
Another task that did not belong to these categories was
feature extraction, including out-of-distribution detec-
tion. This task focuses on the discriminative aspect of
GAN because the studies have directly used GAN archi-
tectures to detect pathologies. Two studies have used
the concept of GAN in ophthalmology image domains
(Table 7). Schlegl et al. proposed an anomaly detection
method using a GAN in the retinal OCT domain [25].
This GAN model estimated the latent space via inverse
mapping learning from the input images and calculated
anomaly scores from the feature space of normal sam-
ples. This anomaly detection architecture has been suc-
cessfully extended to other areas, such as industrial
anomaly detection or chest lesion detection in X-ray
images [10]. Xie et al. built a modified conditional GAN
model for ultra-widefield fundus photography to improve
the detection of retinal diseases [26]. They used an atten-
tion encoder for feature mining in the generator and
designed a multi-branch structure in the discriminator to
extract image features.
Other applications
Here, we address several studies that did not fit our
search criteria, but the applications are noteworthy. A
localization task refers to the identification of a region of
interest rather than a specific pixel segmentation. Zhang
et al. performed retinal pathology localization in fundus
photography using CycleGAN [68]. In this localization
task, the pathological area was detected by subtract-
ing the synthesized normal image from the pathological
image.
Image registration is another task finding the geomet-
ric transformation to structurally align images. It is also
important in automated analysis of multimodal image
analysis such as domain transfer. For example, a data-
set for conditional GAN requires additional image reg-
istration of aligned image pairs for successful training.
Table 5 Summary of literature review for domain transfer task using GAN in ophthalmology imaging domains
GAN = generative adversarial network; OCT = optical coherence tomography
Publication Basic technique Domain Summary
Costa et al. [58] Conditional GAN Vessel image→Fundus photography The study proposed a vessel network to retinal image
translation framework producing simplified vessel
tree and realistic retinal images by estimating latent
space. Autoencoder was used to synthesize new
retinal vessel images apart from training of GAN
Zhao et al. [59] Conditional GAN Vessel image→Fundus photography Retinal image synthesis can be effectively learned in
a data-driven fashion from a relatively small sample
size using a conditional GAN architecture
Yu et al. [60] Pix2pix (with
ResU-net genera-
tor) (conditional
GAN)
Vessel image→Fundus photography To enlarge training datasets for facilitating medical
image analysis, the multiple-channels-multiple-
landmarks (MCML) was developed to synthesize
color fundus images from a combination of vessel
and optic disc masked images
Wu et al. [61] Conditional GAN Volumetric retinal OCT→Fundus autofluorescence The en-face OCT images were synthesized from
volumetric retinal OCT by restricted summed voxel
projection. The fundus autofluorescence images
were generated from en-face OCT images using GAN
to identify the geographic atrophy region
Tavakkoli et al. [62] Conditional GAN Fundus photography→Fluorescein angiography The proposed GAN produced anatomically accurate
fluorescein angiography images that were indistin-
guishable from real angiograms
Yoo et al. [63] CycleGAN Ultra-widefield fundus photography→Fundus
photography
Ultra-widefield images were successfully translated
into traditional fundus photography-style images by
CycleGAN, and the main structural information of the
retina and optic nerve was retained
Ju et al. [64] CycleGAN Fundus photography→Ultra-widefield fundus
photography
The CycleGAN model transferred the color fundus
photographs to ultra-widefield images to introduce
additional data for existing limited ultra-widefield
images. The proposed method was adopted for
diabetic retinopathy grading and lesion detection
Lazaridis et al. [91, 108] Wasserstein
GAN+percep-
tual loss (condi-
tional GAN)
Time-domain OCT→spectral-domain OCT Time-domain OCT was converted to synthetic spec-
tral-domain OCT using GAN. The model improved
the statistical power of the measurements when
compared with those derived from the original OCT
13. Page 13 of 19
You et al. Eye and Vision (2022) 9:6
Mahapatra et al. showed that an autoencoder based on
GAN architecture provided better registration perfor-
mance for fundus photography and retinal angiography
images [69].
Current limitations of GAN techniques
We found that GAN has several limitations that research-
ers should take note of (Fig. 4). First, mode collapse,
which is a phenomenon that continues to output the
same results, is a well-known problem of GAN [70]. To
avoid this failure caused by a model stuck in a local mini-
mum, more variant training data or additional data aug-
mentation techniques are needed. Many GAN models
were trained with no guarantee of convergence. Second,
spatial deformities frequently occur when there are small
training images without spatial alignment. In particular,
in domain transfer using conditional GAN, paired images
with structural and spatial alignment are critically chal-
lenging and require additional image registration in a
preprocessing to obtain high-quality medical images [57,
63]. Third, unintended changes could occur in image-to-
image translation because of the different data distribu-
tions of the structural features between the two image
domains. For example, if only one domain contains many
images with glaucoma in a domain transfer task, Cycle-
GAN can produce glaucomatous changes during image
synthesis. Noise can be represented as unintended out-
liers/confounders and unintended fake features can be
generated from noise for a generator in several GAN
models [26]. Fourth, high-frequency noises and artifacts
of checkerboard patterns are often detected in images
synthesized by a generator with deconvolution [53].
Novel techniques have been developed to reduce noise
and artifacts [71].
GAN and its variants generally consist of two or more
deep learning modules, for example, two generators
and two discriminators in CycleGAN, and thus training
GAN tends to be unstable compared to a single deep
learning module [15]. A problem of vanishing gradients
may also occur if the discriminator performs well, and
the generator learns too slowly. Therefore, tuning the
hyperparameters is sometimes important, and training
can be stopped early to obtain better synthetic images.
However, the occurrence of these problems depends on
the amount of data and the distribution of embedded
pixels and is unpredictable.
In our experience, most of the problems from train-
ing GAN models are solved to some extent by increas-
ing the amount of clinical data extracted from a variety
of patients if the technique is appropriately selected.
Although GAN is widely used for data augmentation,
several previous studies using GAN for image-to-image
translation also suffered from small amounts of data,
similar to other deep learning algorithms [45]. As novel
algorithms are emerging to solve these problems [72],
GAN will eventually be easy to use for image process-
ing in ophthalmology.
Additionally, there is no standard metric for evalu-
ating the performance of GAN for realistic image syn-
thesis [73]. It now relies on subjective judgments from
researchers and clinicians relevant to ophthalmology
imaging [40]. Previous studies adopted classic image
similarity indices, such as mean squared error, mean
absolute error, and structural similarity index [15], but
they are unable to evaluate the realism of synthetic
images. Several studies have shown an improvement in
the diagnostic performance of machine learning after
GAN-based data augmentation [43, 46], but it does not
Table 6 Summary of literature review for post-intervention prediction task using GAN in ophthalmology imaging domains
GAN = generative adversarial network; OCT = optical coherence tomography
Publication Basic technique Domain Intervention Summary
Yoo et al. [15] Conditional GAN, CycleGAN Periorbital facial images Orbital decompression surgery The developed model trans-
formed preoperative facial input
images into predicted postopera-
tive images for orbital decom-
pression for thyroid-associated
ophthalmopathy
Liu et al. [66] Pix2pix (conditional GAN) Retinal OCT Intravitreal anti-vascular
endothelial growth factor
injection
The model generated individual-
ized post-therapeutic OCT images
that could predict the short-term
response of treatment for age-
related macular degeneration
Lee et al. [67] Conditional GAN (multi-channel
inputs)
Retinal OCT (with fluorescein
angiography and indocyanine
green angiography)
Intravitreal anti-vascular
endothelial growth factor
injection
The trained model generated
post-treatment optical coherence
tomography (OCT) images of
neovascular age-related macular
degeneration
14. Page 14 of 19
You et al. Eye and Vision (2022) 9:6
guarantee that GAN produces realistic images. This
problem arises from the application of GAN not only
in ophthalmology but also in all medical areas [10] and
will continue to be a drawback for using GAN.
Discussion
We surveyed the literature relevant to GAN in oph-
thalmology image domains to guide future studies on
image processing in ocular images. To our knowledge,
this work is the first comprehensive literature review
on the use of GAN techniques in ophthalmology image
domains. Recently, a review of GAN in ophthalmology
was reported, but the scope was limited to image synthe-
sis in fundus photography and OCT [73]. GAN research
has thrived in the medical field because machine learn-
ing is data-hungry to achieve a more accurate diagnosis.
In this review, we highlighted the various uses of GAN
in that it can perform segmentation, data augmentation,
denoising, domain transfer, super-resolution, post-inter-
vention prediction, and feature extraction. The num-
ber of publications relevant to this field has also grown
consistently as GAN techniques have become popular
among researchers. We found that GAN can be applied
to most ophthalmology image domains in the literature.
As imaging plays a crucial role in ophthalmology, GAN-
based image synthesis and image-to-image translation
will be highly valuable in improving the quantitative and
personalized evaluation of ocular disorders. Despite the
increasing use of GAN techniques, we also found that it
faces challenges for adaptation to clinical settings.
Recently, a previous paper suggested that the utility of
GAN in image synthesis is unclear for ophthalmology
imaging [73]. However, GAN techniques have shown
better performance in the fields of radiology and pathol-
ogy than other generative deep learning models, such
as autoencoders, fully convolutional networks (FCNs),
and U-nets [74, 75]. In the anomaly detection task for
retinal OCT, GAN models including AnoGAN, Pix2pix,
and CycleGAN outperformed a traditional autoencoder
model, which simply learns latent coding of unlabeled
image data [76]. FCN and U-Net are well-established
deep generative models for detection and segmentation
tasks for biomedical imaging domains [77]. As these do
not consider the detailed features of the output images,
the GAN framework can improve the image synthesis
performance of the FCN and U-Net models [78]. A pre-
vious study on retinal vessel segmentation showed that
conditional GAN outperformed U-Net and other genera-
tive techniques [28]. Given this trend, GAN is expected
to improve image analysis technologies in various tasks.
A more accurate comparison and benchmarking of GAN
techniques will be enabled by future studies and more
clinical data.
The proper selection of the GAN technique and statis-
tical modeling of ocular imaging will improve the perfor-
mance of each image analysis. In this review, we found
that a broad range of custom architectures from GAN
variants was used for different tasks. There is no evidence
of a particularly superior GAN technique. Researchers
can analyze ocular images by newly defining the cus-
tom objective functions of the GAN to fit the specific
task and domain. For example, Cheong et al. modified
the conditional GAN model to effectively denoise OCT
images using a custom loss function including content,
style, total variation, and shadow losses [16]. Moreover,
researchers can incorporate prior information about each
imaging domain to develop GAN models for specific
tasks. For example, researchers have suggested several
statistical distributions of retinal structures and noise
modeling in OCT images [79]. Statistical modeling using
Table 7 Summary of literature review for feature extraction task using GAN in ophthalmology imaging domains
GAN = generative adversarial network; OCT = optical coherence tomography
Publication Basic technique Domain Target Summary
Schlegl et al. [25] f-AnoGAN (Wasserstein
GAN+latent space mapping)
Retinal OCT Intra-retinal fluid detection
(OCT anomaly detection)
The GAN based unsupervised
learning of healthy training data
was trained with fast mapping
from images to encodings in the
latent space. Anomalies were
detected via a combined anomaly
score based on an image recon-
struction error
Xie et al. [26] Conditional GAN (with attention
encoder and multi-branch
structure)
Ultra-widefield fundus
photography (scanning laser
ophthalmoscopy)
Features for retinal diseases The GAN based on the atten-
tion encoder and multi-branch
structure was used to extract
features for retinal disease detec-
tion. The discriminator in GAN was
modified to build the classifier to
detect the disease images
15. Page 15 of 19
You et al. Eye and Vision (2022) 9:6
prior information improved the performance of segmen-
tation of retinal layers and detection of diabetic retin-
opathy in OCT [80]. Since GANs are also mathematical
models for learning the statistical relationship of distribu-
tions of training and target data [6], statistical modeling
using a prior domain knowledge is expected to improve
GAN performance. To adopt this concept in GAN for
medical imaging, various mathematical attempts and val-
idations are needed in future studies.
Several studies have shown that GAN can be a good
choice in overcoming data shortages and lack of large
annotated datasets in ophthalmology [81]. Burlina et al.
showed that a deep learning model trained with only syn-
thetic retinal images generated by PGGAN performed
worse than those trained with real retinal images (0.9706
vs. 0.9235 considering the area under the receiver oper-
ating characteristic curve) [40]. However, several stud-
ies have shown that machine learning models trained
with data integrating both real and GAN-based synthetic
images can outperform those trained with real images in
retinal OCT [43], anterior segment OCT [82], ocular sur-
face image [46], and corneal topography [47]. GAN was
also used for data augmentation of OCT images with rare
retinal diseases in a semi-supervised learning manner
[45]. Studies have shown that GAN-based data augmen-
tation can provide a tool to solve an over-fitting prob-
lem in imbalanced datasets owing to the lack of available
pathological samples. The image synthesis ability of GAN
also provides patient privacy because synthetic images
preserve characteristics as they become unidentifiable.
The synthetic data preserve the manifold in the feature
space of the original dataset [83]. It might be possible that
machine learning researchers release the synthetic data-
set generated by GAN instead of a real dataset to dem-
onstrate their model if there is a problem with patient
privacy. Additionally, the annotation of the dataset can be
incomplete and inaccurate because it is time-consuming
and laborious. According to a previous report regarding
cell segmentation in microscopy images, GAN can be a
solution for this weak annotation problem [84].
Studies using image-to-image translation frameworks
of GAN have focused on segmentation, domain trans-
fer, denoising, super-resolution, and post-intervention
prediction. The denoising function of the GAN may be
effective in decreasing the effect of adversarial attacks
in ophthalmology image domains [85]. Recently devel-
oped GAN architectures, such as Pix2pix and CycleGAN,
have been widely applied in medical image domains
[27]. However, these techniques require spatial align-
ment between the two image domains to obtain high-
quality results. Therefore, additional image registration is
required before the GAN performs a domain transforma-
tion [57]. If the structures in the images are not aligned,
the GAN may perform an image-to-image translation
with deformed results in synthetic images. In this review,
we found that only one study performed image registra-
tion to align the retinal structures between classic fundus
photography and ultra-widefield fundus photography to
improve the performance of CycleGAN [63]. We antici-
pate that this image alignment issue for training GAN
models will be highlighted when data from various cent-
ers measured from multiple devices are collected. More
research is needed, but recent studies have shown that
GAN can also provide solutions to this image alignment
issue [69, 86].
As the deep learning techniques associated with
GAN have been developed, the scope of medical image
processing is rapidly expanding. For example, when
GAN was first introduced, simple vessel segmentation
was the most frequent application of GAN. The recent
work of Tavakkoli et al. achieved a significant advance-
ment in the retinal vessel segmentation problem
Fig. 4 Examples of problems encountered using GAN techniques. a Mode collapse where the generator produces limited varieties of samples. b
Spatial deformity due to small training images without spatial alignment. c Unintended changes due to the difference of data distribution between
two domains. d Checker-board artifacts in synthetic images. All of the images were generated according to publicly available datasets and the
standard GAN methods
16. Page 16 of 19
You et al. Eye and Vision (2022) 9:6
because their conditional GAN model provides real-
istic retinal fluorescein angiography, which can be
used in a clinical setting [62]. Novel image process-
ing techniques using unpaired image datasets, such as
CycleGAN, have extended the range of training image
domains, so ocular images from more diverse modali-
ties are expected to be used for developing AI sys-
tems [38, 63]. Multi-domain GAN models that handle
images from various domains at once have also been
developed to fuse data for more accurate diagnosis.
For example, Lee et al. reported a conditional GAN
using multi-domain inputs analyzing OCT and fluo-
rescein angiography to predict more accurate post-
treatment retinal state prediction [67]. In the future,
new GAN techniques such as StyleGAN [13], which is
excellent at extracting and transforming features, and
StarGAN [23], which performs multi-domain transfor-
mation, are also expected to be used in the ophthal-
mology imaging domains to solve clinical problems. To
adopt this rapid technical development, future stud-
ies require multidisciplinary (clinician–engineer) col-
laboration and collection of more multi-domain ocular
images. Clinicians need to feedback to engineers to
improve the technical completeness of GAN.
Most studies included in this review used train-
ing and validation datasets extracted from the same
study group. We found that no clinical trials have been
conducted that explored the use of GAN. Machine
learning techniques, including GAN, do not guaran-
tee performance in external datasets independent of
training sets. It has not been confirmed whether data
augmentation through GAN can increase the diag-
nostic accuracy of AI systems for ocular diseases in
real clinics. A GAN can be used to bridge the domain
gap between training and external data from different
sources [64]. If difficult access to reliable annotated
data from multiple data sources remains problem-
atic, domain adaptation can be considered to address
the generalization issue [87]. Domain adaptation via
the domain transfer function of a GAN may provide a
chance to use a machine learning system in different
settings. For example, retinal images taken with ultra-
widefield fundus photography can be analyzed by an
AI system developed with FP via domain transfer using
GAN [63, 64]. Although GAN has several shortcom-
ings, its ability to adapt to domains and expand data
by generating realistic images can increase generaliz-
ability and may help to increase the use of machine
learning algorithms in ophthalmology image domains.
However, further studies are required to determine
whether the application of GAN techniques will
improve the diagnostic performance of machine learn-
ing models in real world clinical situations.
Conclusion
The findings of this work suggest that the direction
of deep learning research in ophthalmology has ben-
efited from GAN. GAN techniques have established
an extension of datasets and modalities in ophthal-
mology. The adoption of GAN in ophthalmology is
still in its early stages of clinical validation compared
with deep learning classification techniques because
several problems need to be overcome for practical
use. However, the proper selection of a GAN tech-
nique and statistical modeling of ocular imaging will
improve the performance of each image analysis. We
hope that this review will fuel more studies using
GAN in ophthalmology image domains. More accu-
rate algorithms for the detection of pathological oph-
thalmic conditions would be enabled by selection of
proper GAN techniques by maximizing the potential
of ophthalmology datasets.
Acknowledgements
Not applicable.
Authors’contributions
AY conceived the idea, interpreted the literature, and drafted the manuscript.
JJK reviewed the manuscript critically. IHR interpreted the several studies and
revised the manuscript. TKY designed the study, interpreted the literature, and
reviewed the manuscript. All authors read and approved the final manuscript.
Funding
This research did not receive any specific grant from funding agencies in the
public, commercial, or not-for-profit sectors.
Availability of data and materials
Data sharing is not applicable to this article as no datasets were generated or
analyzed during the current study.
Declarations
Ethics approval and consent to participate
Not applicable. Ethical approval was not sought, as the study was based
entirely on previously published data. All procedures were performed in
accordance with the ethical standards of the 1964 Helsinki Declaration and its
amendments.
Consent for publication
Not applicable.
Competing interests
Jin Kuk Kim and Ik Hee Ryu are executives of VISUWORKS, Inc., which is a
Korean Artificial Intelligence company providing medical machine learning
solutions. Jin Kuk Kim is also an executive of the Korea Intelligent Medical
Industry Association. They received salaries or stocks as part of the standard
compensation package. The remaining authors declare no conflict of interest.
Author details
1
School of Architecture, Kumoh National Institute of Technology, Gumi, Gyeo-
ngbuk, South Korea. 2
B&VIIT Eye Center, Seoul, South Korea. 3
VISUWORKS,
Seoul, South Korea. 4
Department of Ophthalmology, Aerospace Medical
Center, Republic of Korea Air Force, 635 Danjae‑ro, Namil‑myeon, Cheong-
won‑gun, Cheongju, Chungcheongbuk‑do 363‑849, South Korea.
Received: 15 July 2021 Accepted: 11 January 2022
17. Page 17 of 19
You et al. Eye and Vision (2022) 9:6
References
1. Wang W, Yan W, Müller A, Keel S, He M. Association of socioeconomics
with prevalence of visual impairment and blindness. JAMA Ophthalmol.
2017;135(12):1295–302.
2. Liu H, Li L, Wormstone IM, Qiao C, Zhang C, Liu P, et al. Development
and validation of a deep learning system to detect glaucomatous
optic neuropathy using fundus photographs. JAMA Ophthalmol.
2019;137(12):1353–60.
3. Khan SM, Liu X, Nath S, Korot E, Faes L, Wagner SK, et al. A global
review of publicly available datasets for ophthalmological imaging:
barriers to access, usability, and generalisability. Lancet Digit Health.
2021;3(1):e51-66.
4. De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Black-
well S, et al. Clinically applicable deep learning for diagnosis and referral
in retinal disease. Nat Med. 2018;24(9):1342–50.
5. Creswell A, White T, Dumoulin V, Arulkumaran K, Sengupta B, Bharath
AA. Generative adversarial networks: an overview. IEEE Signal Process
Mag. 2018;35:53–65.
6. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S,
et al. Generative adversarial nets. In: Proceedings of the 27th Interna-
tional Conference on Neural Information Processing Systems. Montreal,
Canada; p. 2672–2680.
7. Barbedo JGA. Impact of dataset size and variety on the effectiveness
of deep learning and transfer learning for plant disease classification.
Comput Electron Agric. 2018;153:46–53.
8. Sorin V, Barash Y, Konen E, Klang E. Creating artificial images for radiol-
ogy applications using generative adversarial networks (GANs)-a
systematic review. Acad Radiol. 2020;27(8):1175–85.
9. Wolterink JM, Mukhopadhyay A, Leiner T, Vogl TJ, Bucher AM, Išgum I.
Generative adversarial networks: a primer for radiologists. Radiograph-
ics. 2021;41(3):840–57.
10. Yi X, Walia E, Babyn P. Generative adversarial network in medical imag-
ing: a review. Med Image Anal. 2019;58:101552.
11. Tsiknakis N, Theodoropoulos D, Manikis G, Ktistakis E, Boutsora O,
Berto A, et al. Deep learning for diabetic retinopathy detection and
classification based on fundus images: a review. Comput Biol Med.
2021;135:104599.
12. Abdelhalim ISA, Mohamed MF, Mahdy YB. Data augmentation for skin
lesion using self-attention based progressive generative adversarial
network. Expert Syst Appl. 2021;165:113922.
13. Fetty L, Bylund M, Kuess P, Heilemann G, Nyholm T, Georg D, et al.
Latent space manipulation for high-resolution medical image synthesis
via the StyleGAN. Z Med Phys. 2020;30(4):305–14.
14. Mirza M, Osindero S. Conditional generative adversarial nets.
arXiv:1411.1784. 2014.
15. Yoo TK, Choi JY, Kim HK. A generative adversarial network approach
to predicting postoperative appearance after orbital decompression
surgery for thyroid eye disease. Comput Biol Med. 2020;118:103628.
16. Cheong H, Devalla SK, Pham TH, Zhang L, Tun TA, Wang X, et al. Deshad-
owGAN: a deep learning approach to remove shadows from optical
coherence tomography images. Transl Vis Sci Technol. 2020;9(2):23.
17. Zhou Y, Wang B, He X, Cui S, Shao L. DR-GAN: conditional generative
adversarial network for fine-grained lesion synthesis on diabetic retin-
opathy images. IEEE J Biomed Health Inform. 2020. https://doi.org/10.
1109/JBHI.2020.3045475.
18. Wang W, Li X, Xu Z, Yu W, Zhao J, Ding D, et al. Learning two-
stream CNN for multi-modal age-related macular degeneration
categorization. arXiv:2012.01879.
19. Isola P, Zhu JY, Zhou T, Efros AA. Image-to-image translation with con-
ditional adversarial networks. In: Proceedings of the IEEE conference on
computer vision and pattern recognition. 2017. p. 1125–34.
20. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, et al. ESRGAN: enhanced super-
resolution generative adversarial networks. arXiv:180900219.
21. Zhu JY, Park T, Isola P, Efros AA. Unpaired image-to-image translation
using cycle-consistent adversarial networks. In: Proceedings of the
IEEE international conference on computer vision. 2017. p. 2223–32.
22. Manakov I, Rohm M, Kern C, Schworm B, Kortuem K, Tresp V, et al.
Noise as domain shift: denoising medical images by unpaired image
translation. In: Wang Q, Milletari F, Nguyen HV, Albarqouni S, Cardoso
MJ, Rieke N, et al., editors. Domain adaptation and representation
transfer and medical image learning with less labels and imperfect
data. Cham: Springer International Publishing; 2019. p. 3–10.
23. Choi Y, Choi M, Kim M, Ha JW, Kim S, Choo J. StarGAN: unified genera-
tive adversarial networks for multi-domain image-to-image transla-
tion. In: 2018 IEEE/CVF conference on computer vision and pattern
recognition. 2018. p. 8789–97.
24. Lee D, Moon WJ, Ye JC. Assessing the importance of magnetic reso-
nance contrasts using collaborative generative adversarial networks.
Nat Mach Intell. 2020;2:34–42.
25. Schlegl T, Seeböck P, Waldstein SM, Langs G, Schmidt-Erfurth U.
f-AnoGAN: fast unsupervised anomaly detection with generative
adversarial networks. Med Image Anal. 2019;54:30–44.
26. Xie H, Lei H, Zeng X, He Y, Chen G, Elazab A, et al. AMD-GAN: atten-
tion encoder and multi-branch structure based generative adver-
sarial networks for fundus disease detection from scanning laser
ophthalmoscopy images. Neural Netw. 2020;132:477–90.
27. Iqbal T, Ali H. Generative adversarial network for medical images (MI-
GAN). J Med Syst. 2018;42:231.
28. Son J, Park SJ, Jung KH. Towards accurate segmentation of retinal
vessels and the optic disc in fundoscopic images with generative
adversarial networks. J Digit Imaging. 2019;32(3):499–512.
29. Rammy SA, Abbas W, Hassan NU, Raza A, Zhang W. CPGAN: Condi-
tional patch-based generative adversarial network for retinal vessel
segmentation. IET Image Process. 2019;14(6):1081–90.
30. Park KB, Choi SH, Lee JY. M-GAN: retinal blood vessel segmenta-
tion by balancing losses through stacked deep fully convolutional
networks. IEEE Access. 2020;8:146308–22.
31. Yang J, Dong X, Hu Y, Peng Q, Tao G, Ou Y, et al. Fully automatic
arteriovenous segmentation in retinal images via topology-aware
generative adversarial networks. Interdiscip Sci. 2020;12(3):323–34.
32. Zhao H, Qiu X, Lu W, Huang H, Jin X. High-quality retinal vessel
segmentation using generative adversarial network with a large
receptive field. Int J Imaging Sys Technol. 2020;30:828–42.
33. Wang S, Yu L, Yang X, Fu CW, Heng PA. Patch-based output space
adversarial learning for joint optic disc and cup segmentation. IEEE
Trans Med Imaging. 2019;38(11):2485–95.
34. Kadambi S, Wang Z, Xing E. WGAN domain adaptation for the joint
optic disc-and-cup segmentation in fundus images. Int J Comput
Assist Radiol Surg. 2020;15(7):1205–13.
35. Bian X, Luo X, Wang C, Liu W, Lin X. Optic disc and optic cup segmen-
tation based on anatomy guided cascade network. Comput Methods
Programs Biomed. 2020;197:105717.
36. Heisler M, Bhalla M, Lo J, Mammo Z, Lee S, Ju MJ, et al. Semi-super-
vised deep learning based 3D analysis of the peripapillary region.
Biomed Opt Express. 2020;11(7):3843–56.
37. Yildiz E, Arslan AT, Yildiz Tas A, Acer AF, Demir S, Sahin A, et al. Genera-
tive adversarial network based automatic segmentation of corneal
sub basal nerves on in vivo confocal microscopy images. Transl Vis
Sci Technol. 2021;10(6):33.
38. Khan ZK, Umar AI, Shirazi SH, Rasheed A, Qadir A, Gul S. Image
based analysis of meibomian gland dysfunction using conditional
generative adversarial neural network. BMJ Open Ophthalmol.
2021;6(1):e000436.
39. Diaz-Pinto A, Colomer A, Naranjo V, Morales S, Xu Y, Frangi AF. Retinal
image synthesis and semi-supervised learning for glaucoma assess-
ment. IEEE Trans Med Imaging. 2019;38(9):2211–8.
40. Burlina PM, Joshi N, Pacheco KD, Liu TYA, Bressler NM. Assessment of
deep generative models for high-resolution synthetic retinal image
generation of age-related macular degeneration. JAMA Ophthalmol.
2019;137(3):258–64.
41. Wang S, Wang X, Hu Y, Shen Y, Yang Z, Gan M, et al. Diabetic retin-
opathy diagnosis using multichannel generative adversarial network
with semisupervision. IEEE Trans Autom Sci Eng. 2021;18:574–85.
42. He X, Fang L, Rabbani H, Chen X, Liu Z. Retinal optical coherence
tomography image classification with label smoothing generative
adversarial network. Neurocomputing. 2020;405:37–47.
43. Zheng C, Xie X, Zhou K, Chen B, Chen J, Ye H, et al. Assessment of
generative adversarial networks model for synthetic optical coher-
ence tomography images of retinal disorders. Transl Vis Sci Technol.
2020;9(2):29.
18. Page 18 of 19
You et al. Eye and Vision (2022) 9:6
44. Kugelman J, Alonso-Caneiro D, Read SA, Vincent SJ, Chen FK, Collins
MJ. Data augmentation for patch-based OCT chorio-retinal segmenta-
tion using generative adversarial networks. Neural Comput & Applic.
2021;33:7393–408.
45. Yoo TK, Choi JY, Kim HK. Feasibility study to improve deep learning in
OCT diagnosis of rare retinal diseases with few-shot classification. Med
Biol Eng Comput. 2021;59(2):401–15.
46. Yoo TK, Choi JY, Kim HK, Ryu IH, Kim JK. Adopting low-shot deep learn-
ing for the detection of conjunctival melanoma using ocular surface
images. Comput Methods Programs Biomed. 2021;205:106086.
47. Abdelmotaal H, Abdou AA, Omar AF, El-Sebaity DM, Abdelazeem K.
Pix2pix conditional generative adversarial networks for scheimpflug
camera color-coded corneal tomography image generation. Trans Vis
Sci Tech. 2021;10(7):21–21.
48. Halupka KJ, Antony BJ, Lee MH, Lucy KA, Rai RS, Ishikawa H, et al. Retinal
optical coherence tomography image enhancement via deep learning.
Biomed Opt Express. 2018;9(12):6205–21.
49. Huang Y, Lu Z, Shao Z, Ran M, Zhou J, Fang L, et al. Simultaneous
denoising and super-resolution of optical coherence tomography
images based on generative adversarial network. Opt Express.
2019;27(9):12289–307.
50. Chen Z, Zeng Z, Shen H, Zheng X, Dai P, Ouyang P. DN-GAN: Denois-
ing generative adversarial networks for speckle noise reduction in
optical coherence tomography images. Biomed Signal Process Control.
2020;55:101632.
51. Ouyang J, Mathai TS, Lathrop K, Galeotti J. Accurate tissue interface
segmentation via adversarial pre-segmentation of anterior segment
OCT images. Biomed Opt Express. 2019;10(10):5291–324.
52. Das V, Dandapat S, Bora PK. Unsupervised super-resolution of OCT
images using generative adversarial network for improved age-related
macular degeneration diagnosis. IEEE Sens J. 2020;20:8746–56.
53. Yoo TK, Choi JY, Kim HK. CycleGAN-based deep learning technique
for artifact reduction in fundus photography. Graefes Arch Clin Exp
Ophthalmol. 2020;258(8):1631–7.
54. Luo Y, Chen K, Liu L, Liu J, Mao J, Ke G, et al. Dehaze of cataractous
retinal images using an unpaired generative adversarial network. IEEE J
Biomed Health Inform. 2020;24(12):3374–83.
55. Mahapatra D, Bozorgtabar B, Garnavi R. Image super-resolution using
progressive generative adversarial networks for medical image analysis.
Comput Med Imaging Graph. 2019;71:30–9.
56. Ha A, Sun S, Kim YK, Lee J, Jeoung JW, Kim HC, et al. Deep-
learning-based enhanced optic-disc photography. PLoS One.
2020;15(10):e0239913.
57. Shin Y, Yang J, Lee YH. Deep generative adversarial networks: applica-
tions in musculoskeletal imaging. Radiol Artif Intell. 2021;3(3):e200157.
58. Costa P, Galdran A, Meyer MI, Niemeijer M, Abramoff M, Mendonca AM,
et al. End-to-end adversarial retinal image synthesis. IEEE Trans Med
Imaging. 2018;37(3):781–91.
59. Zhao H, Li H, Maurer-Stroh S, Cheng L. Synthesizing retinal and
neuronal images with generative adversarial nets. Med Image Anal.
2018;49:14–26.
60. Yu Z, Xiang Q, Meng J, Kou C, Ren Q, Lu Y. Retinal image synthesis from
multiple-landmarks input with generative adversarial networks. Biomed
Eng Online. 2019;18(1):62.
61. Wu M, Cai X, Chen Q, Ji Z, Niu S, Leng T, et al. Geographic atrophy
segmentation in SD-OCT images using synthesized fundus autofluores-
cence imaging. Comput Methods Programs Biomed. 2019;182:105101.
62. Tavakkoli A, Kamran SA, Hossain KF, Zuckerbrod SL. A novel deep learn-
ing conditional generative adversarial network for producing angiogra-
phy images from retinal fundus photographs. Sci Rep. 2020;10(1):21580.
63. Yoo TK, Ryu IH, Kim JK, Lee IS, Kim JS, Kim HK, et al. Deep learning can
generate traditional retinal fundus photographs using ultra-widefield
images via generative adversarial networks. Comput Methods Pro-
grams Biomed. 2020;197:105761.
64. Ju L, Wang X, Zhao X, Bonnington P, Drummond T, Ge Z. Leveraging
regular fundus images for training UWF fundus diagnosis models via
adversarial learning and pseudo-labeling. IEEE Trans Med Imaging.
2021;40(10):2911–25.
65. Liu TYA, Farsiu S, Ting DS. Generative adversarial networks to predict
treatment response for neovascular age-related macular degeneration:
interesting, but is it useful? Br J Ophthalmol. 2020;104(12):1629–30.
66. Liu Y, Yang J, Zhou Y, Wang W, Zhao J, Yu W, et al. Prediction of OCT
images of short-term response to anti-VEGF treatment for neovascular
age-related macular degeneration using generative adversarial net-
work. Br J Ophthalmol. 2020;104(12):1735–40.
67. Lee H, Kim S, Kim MA, Chung H, Kim HC. Post-treatment prediction
of optical coherence tomography using a conditional generative
adversarial network in age-related macular degeneration. Retina.
2021;41(3):572–80.
68. Zhang Z, Ji Z, Chen Q, Fan W, Yuan S. Joint optimization of CycleGAN
and CNN classifier for detection and localization of retinal patholo-
gies on color fundus photographs. IEEE J Biomed Health Inform. 2021.
https://doi.org/10.1109/JBHI.2021.3092339.
69. Mahapatra D, Ge Z. Training data independent image registration
using generative adversarial networks and domain adaptation. Pattern
Recogn. 2020;100:107109.
70. Srivastava A, Valkov L, Russell C, Gutmann MU, Sutton C. Veegan: Reduc-
ing mode collapse in gans using implicit variational learning. In: Pro-
ceedings of the 31st International Conference on Neural Information
Processing System. 2017. p. 3310–20. https://doi.org/10.5555/3294996.
3295090.
71. Lee OY, Shin YH, Kim JO. Multi-perspective discriminators-based
generative adversarial network for image super resolution. IEEE Access.
2019;7:136496–510.
72. Liu MY, Huang X, Mallya A, Karras T, Aila T, Lehtinen J, et al. Few-shot
unsupervised image-to-image translation. In: Proceedings of the IEEE
International Conference on Computer Vision. 2019. p. 10551–60.
73. Wang Z, Lim G, Ng WY, Keane PA, Campbell JP, Tan GSW, et al. Genera-
tive adversarial networks in ophthalmology: what are these and how
can they be used? Curr Opin Ophthalmol. 2021;32(5):459–67.
74. Tschuchnig ME, Oostingh GJ, Gadermayr M. Generative adversarial
networks in digital pathology: a survey on trends and future potential.
Patterns (N Y). 2020;1(6):100089.
75. Kearney V, Ziemer BP, Perry A, Wang T, Chan JW, Ma L, et al. Atten-
tion-aware discrimination for MR-to-CT image translation using
cycle-consistent generative adversarial networks. Radiol Artif Intell.
2020;2(2):e190027.
76. Zhou K, Xiao Y, Yang J, Cheng J, Liu W, Luo W, et al. Encoding structure-
texture relation with P-Net for anomaly detection in retinal images. In:
Vedaldi A, Bischof H, Brox T, Frahm J-M, editors. Computer vision—ECCV
2020. Cham: Springer International Publishing; 2020. p. 360–77.
77. Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for
biomedical image segmentation. In: Navab N, Hornegger J, Wells WM,
Frangi AF, editors. Medical image computing and computer-assisted
intervention—MICCAI 2015. Cham: Springer International Publishing;
2015. p. 234–41.
78. Lei B, Xia Z, Jiang F, Jiang X, Ge Z, Xu Y, et al. Skin lesion segmentation
via generative adversarial networks with dual discriminators. Med
Image Anal. 2020;64:101716.
79. Jorjandi S, Amini Z, Plonka G, Rabbani H. Statistical modeling of retinal
optical coherence tomography using the Weibull mixture model.
Biomed Opt Express. 2021;12(9):5470–88.
80. Grzywacz NM, de Juan J, Ferrone C, Giannini D, Huang D, Koch G, et al.
Statistics of optical coherence tomography data from human retina.
IEEE Trans Med Imaging. 2010;29(6):1224–37.
81. Bellemo V, Burlina P, Yong L, Wong TY, Ting DSW. Generative adversarial
networks (GANs) for retinal fundus image synthesis. In: Carneiro G, You
S, editors. Computer vision—ACCV 2018 Workshops. Cham: Springer
International Publishing; 2019. p. 289–302.
82. Zheng C, Bian F, Li L, Xie X, Liu H, Liang J, et al. Assessment of generative
adversarial networks for synthetic anterior segment optical coherence
tomography images in closed-angle detection. Transl Vis Sci Technol.
2021;10(4):34.
83. Yoon J, Drumright LN, van der Schaar M. Anonymization through data
synthesis using generative adversarial networks (ADS-GAN). IEEE J
Biomed Health Inform. 2020;24(8):2378–88.
84. He J, Wang C, Jiang D, Li Z, Liu Y, Zhang T. CycleGAN with an improved
loss function for cell detection using partly labeled images. IEEE J
Biomed Health Inform. 2020;24(9):2473–80.
85. Yoo TK, Choi JY. Outcomes of adversarial attacks on deep learning
models for ophthalmology imaging domains. JAMA Ophthalmol.
2020;138(11):1213–5.
19. Page 19 of 19
You et al. Eye and Vision (2022) 9:6
• fast, convenient online submission
• thorough peer review by experienced researchers in your field
• rapid publication on acceptance
• support for research data, including large and complex data types
• gold Open Access which fosters wider collaboration and increased citations
maximum visibility for your research: over 100M website views per year
•
At BMC, research is always in progress.
Learn more biomedcentral.com/submissions
Ready to submit your research
Ready to submit your research ? Choose BMC and benefit from:
? Choose BMC and benefit from:
86. Mahapatra D, Antony B, Sedai S, Garnavi R. Deformable medical image
registration using generative adversarial networks. In: 2018 IEEE 15th
International Symposium on Biomedical Imaging (ISBI 2018). 2018. p.
1449–53.
87. Çallı E, Sogancioglu E, van Ginneken B, van Leeuwen KG, Murphy
K. Deep learning for chest X-ray analysis: a survey. Med Image Anal.
2021;72:102125.
88. Frid-Adar M, Diamant I, Klang E, Amitai M, Goldberger J, Greenspan
H. GAN-based synthetic medical image augmentation for increased
CNN performance in liver lesion classification. Neurocomputing.
2018;321:321–31.
89. Onishi Y, Teramoto A, Tsujimoto M, Tsukamoto T, Saito K, Toyama H, et al.
Automated pulmonary nodule classification in computed tomography
images using a deep convolutional neural network trained by genera-
tive adversarial networks. Biomed Res Int. 2019;2019:e6051939.
90. Hu Z, Jiang C, Sun F, Zhang Q, Ge Y, Yang Y, et al. Artifact correction in
low-dose dental CT imaging using Wasserstein generative adversarial
networks. Med Phys. 2019;46(4):1686–96.
91. Lazaridis G, Lorenzi M, Ourselin S, Garway-Heath D. Improving statistical
power of glaucoma clinical trials using an ensemble of cyclical genera-
tive adversarial networks. Med Image Anal. 2021;68:101906.
92. Kim M, Kim S, Kim M, Bae HJ, Park JW, Kim N. Realistic high-resolution
lateral cephalometric radiography generated by progressive grow-
ing generative adversarial network and quality evaluations. Sci Rep.
2021;11(1):12563.
93. Teramoto A, Tsukamoto T, Yamada A, Kiriyama Y, Imaizumi K, Saito
K, et al. Deep learning approach to classification of lung cytological
images: Two-step training using actual and synthesized images by
progressive growing of generative adversarial networks. PLoS One.
2020;15(3):e0229951.
94. Park JE, Eun D, Kim HS, Lee DH, Jang RW, Kim N. Generative adversarial
network for glioblastoma ensures morphologic variations and improves
diagnostic model for isocitrate dehydrogenase mutant type. Sci Rep.
2021;11(1):9912.
95. Zhao C, Shuai R, Ma L, Liu W, Hu D, Wu M. Dermoscopy image
classification based on StyleGAN and DenseNet201. IEEE Access.
2021;9:8659–79.
96. Zhang X, Song H, Zhang K, Qiao J, Liu Q. Single image super-resolution
with enhanced Laplacian pyramid network via conditional generative
adversarial learning. Neurocomputing. 2020;398:531–8.
97. Wang H, Rivenson Y, Jin Y, Wei Z, Gao R, Günaydın H, et al. Deep learning
enables cross-modality super-resolution in fluorescence microscopy.
Nat Methods. 2019;16(1):103–10.
98. Armanious K, Jiang C, Fischer M, Küstner T, Hepp T, Nikolaou K, et al.
MedGAN: Medical image translation using GANs. Comput Med Imag-
ing Graph. 2020;79:101684.
99. Munawar F, Azmat S, Iqbal T, Grönlund C, Ali H. Segmentation of lungs
in chest X-Ray image using generative adversarial networks. IEEE
Access. 2020;8:153535–45.
100. Maspero M, Savenije MHF, Dinkla AM, Seevinck PR, Intven MPW,
Jurgenliemk-Schulz IM, et al. Dose evaluation of fast synthetic-CT
generation using a generative adversarial network for general pelvis
MR-only radiotherapy. Phys Med Biol. 2018;63(18):185001.
101. Moran MBH, Faria MDB, Giraldi GA, Bastos LF, Conci A. Using super-
resolution generative adversarial network models and transfer learning
to obtain high resolution digital periapical radiographs. Comput Biol
Med. 2021;129:104139.
102. Becker AS, Jendele L, Skopek O, Berger N, Ghafoor S, Marcon M, et al.
Injecting and removing suspicious features in breast imaging with
CycleGAN: a pilot study of automated adversarial attacks using neural
networks on small images. Eur J Radiol. 2019;120:108649.
103. Sandfort V, Yan K, Pickhardt PJ, Summers RM. Data augmentation using
generative adversarial networks (CycleGAN) to improve generalizability
in CT segmentation tasks. Sci Rep. 2019;9(1):16884.
104. Yoo TK, Choi JY, Jang Y, Oh E, Ryu IH. Toward automated severe pharyn-
gitis detection with smartphone camera using deep learning networks.
Comput Biol Med. 2020;125:103980.
105. Jafari MH, Girgis H, Van Woudenberg N, Moulson N, Luong C, Fung
A, et al. Cardiac point-of-care to cart-based ultrasound transla-
tion using constrained CycleGAN. Int J Comput Assist Radiol Surg.
2020;15(5):877–86.
106. Yang T, Wu T, Li L, Zhu C. SUD-GAN: deep convolution generative
adversarial network combined with short connection and dense block
for retinal vessel segmentation. J Digit Imaging. 2020;33(4):946–57.
107. Zhou Y, Chen Z, Shen H, Zheng X, Zhao R, Duan X. A refined equilibrium
generative adversarial network for retinal vessel segmentation. Neuro-
computing. 2021;437:118–30.
108. Lazaridis G, Lorenzi M, Mohamed-Noriega J, Aguilar-Munoa S, Suzuki
K, Nomoto H, et al. OCT signal enhancement with deep learning. Oph-
thalmol Glaucoma. 2021;4(3):295–304.