This document provides an overview of a capstone project to develop a machine learning model using convolutional neural networks to detect melanoma in mole images. The goal is to analyze photos taken with a smartphone to aid in early detection of skin cancer. The project will use a dataset of 2,000 labeled images to retrain the Inception-v3 model and evaluate its ability to correctly classify moles as benign or malignant. Key metrics like recall will be used to assess the model's performance, with the aim of achieving above human-level diagnostic accuracy. Data exploration found some inconsistencies that need addressing, such as stickers only present in benign images, to avoid biasing the model.
RECOGNITION OF SKIN CANCER IN DERMOSCOPIC IMAGES USING KNN CLASSIFIERADEIJ Journal
The largest organ of the body is human skin. Melanoma is a fastest growing & deadliest cancer which starts in pigment cells (melanocytes) of the skin that mostly occurs on the exposed parts of the body. Early detection is vital in treating this type of skin cancer but the time and effort required is immense. Dermoscopy is a non invasive skin imaging technique of acquiring a magnified and illuminated image of a region of skin for increased clarity of the spots on the skin The use of machine learning and automation of the process involved in detection will not only save time but will also provide a more accurate diagnosis. The skin images collected from the databases cannot be directly classified by the automation techniques. The reason is twofold: (a) Lack of clarity in the features which is mainly due to the poor contrast of the raw image and (b) Large dimensions of the input image which causes the complexity of the system. Hence, suitable techniques must be adopted prior to the image classification process to overcome these drawbacks. The first drawback can be minimized by adopting suitable pre- processing techniques which can enhance the contrast of the input images. The second drawback is solved by incorporating the feature extraction technique which reduces the dimensions of the input image to high extent. Further, K-NN (K-Nearest Neighbor) classifier is used for classification of the given image into cancerous or non- cancerous.
Skin Cancer Detection using Digital Image Processing and Implementation using...ijtsrd
Melanoma is a serious type of skin cancer. It starts in skin cells called melanocytes. There are 3 main types of skin cancer, Melanoma, Basal and Squamous cell carcinoma. Melanoma is more likely to spread to other parts of the body. Early detection of malignant melanoma in dermoscopy images is very important and critical, since its detection in the early stage can be helpful to cure it. Computer Aided Diagnosis systems can be very helpful to facilitate the early detection of cancers for dermatologists. Image processing is a commonly used method for skin cancer detection from the appearance of affected area on the skin. In this work, a computerised method has been developed to make use of Neural Networks in the field of medical image processing. The ultimate aim of this paper is to implement cost-effective emergency support systems to process the medical images. It is more advantageous to patients. The dermoscopy image of suspect area of skin cancer is taken and it goes under various pre-processing technique for noise removal and image enhancement. Then the image is undergone to segmentation using Thresholding method. Some features of image have to be extracted using ABCD rules. In this work, Asymmetry index and Geometric features are extracted from the segmented image. These features are given as the input to classifier. Artificial Neural Network ANN with feed forward architecture is used for classification purpose. It classifies the given image into cancerous or non-cancerous. The proposed algorithm has been tested on the ISIC International Skin Imaging Collaboration 2017 training and test datasets. The ground truth data of each image is available as well, so performance of this work can evaluate quantitatively. Khaing Thazin Oo | Dr. Moe Mon Myint | Dr. Khin Thuzar Win "Skin Cancer Detection using Digital Image Processing and Implementation using ANN and ABCD Features" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-6 , October 2018, URL: http://www.ijtsrd.com/papers/ijtsrd18751.pdf
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
RECOGNITION OF SKIN CANCER IN DERMOSCOPIC IMAGES USING KNN CLASSIFIERADEIJ Journal
The largest organ of the body is human skin. Melanoma is a fastest growing & deadliest cancer which starts in pigment cells (melanocytes) of the skin that mostly occurs on the exposed parts of the body. Early detection is vital in treating this type of skin cancer but the time and effort required is immense. Dermoscopy is a non invasive skin imaging technique of acquiring a magnified and illuminated image of a region of skin for increased clarity of the spots on the skin The use of machine learning and automation of the process involved in detection will not only save time but will also provide a more accurate diagnosis. The skin images collected from the databases cannot be directly classified by the automation techniques. The reason is twofold: (a) Lack of clarity in the features which is mainly due to the poor contrast of the raw image and (b) Large dimensions of the input image which causes the complexity of the system. Hence, suitable techniques must be adopted prior to the image classification process to overcome these drawbacks. The first drawback can be minimized by adopting suitable pre- processing techniques which can enhance the contrast of the input images. The second drawback is solved by incorporating the feature extraction technique which reduces the dimensions of the input image to high extent. Further, K-NN (K-Nearest Neighbor) classifier is used for classification of the given image into cancerous or non- cancerous.
Skin Cancer Detection using Digital Image Processing and Implementation using...ijtsrd
Melanoma is a serious type of skin cancer. It starts in skin cells called melanocytes. There are 3 main types of skin cancer, Melanoma, Basal and Squamous cell carcinoma. Melanoma is more likely to spread to other parts of the body. Early detection of malignant melanoma in dermoscopy images is very important and critical, since its detection in the early stage can be helpful to cure it. Computer Aided Diagnosis systems can be very helpful to facilitate the early detection of cancers for dermatologists. Image processing is a commonly used method for skin cancer detection from the appearance of affected area on the skin. In this work, a computerised method has been developed to make use of Neural Networks in the field of medical image processing. The ultimate aim of this paper is to implement cost-effective emergency support systems to process the medical images. It is more advantageous to patients. The dermoscopy image of suspect area of skin cancer is taken and it goes under various pre-processing technique for noise removal and image enhancement. Then the image is undergone to segmentation using Thresholding method. Some features of image have to be extracted using ABCD rules. In this work, Asymmetry index and Geometric features are extracted from the segmented image. These features are given as the input to classifier. Artificial Neural Network ANN with feed forward architecture is used for classification purpose. It classifies the given image into cancerous or non-cancerous. The proposed algorithm has been tested on the ISIC International Skin Imaging Collaboration 2017 training and test datasets. The ground truth data of each image is available as well, so performance of this work can evaluate quantitatively. Khaing Thazin Oo | Dr. Moe Mon Myint | Dr. Khin Thuzar Win "Skin Cancer Detection using Digital Image Processing and Implementation using ANN and ABCD Features" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-2 | Issue-6 , October 2018, URL: http://www.ijtsrd.com/papers/ijtsrd18751.pdf
International Journal of Computational Engineering Research (IJCER) is dedicated to protecting personal information and will make every reasonable effort to handle collected information appropriately. All information collected, as well as related requests, will be handled as carefully and efficiently as possible in accordance with IJCER standards for integrity and objectivity.
Computer Vision for Skin Cancer Diagnosis and Recognition using RBF and SOMCSCJournals
Human skin is the largest organ in our body which provides protection against heat, light, infections and injury. It also stores water, fat, and vitamin. Cancer is the leading cause of death in economically developed countries and the second leading cause of death in developing countries. Skin cancer is the most commonly diagnosed type of cancer among men and women. Exposure to UV rays, modernize diets, smoking, alcohol and nicotine are the main cause. Cancer is increasingly recognized as a critical public health problem in Ethiopia. There are three type of skin cancer and they are recognized based on their own properties. In view of this, a digital image processing technique is proposed to recognize and predict the different types of skin cancers using digital image processing techniques. Sample skin cancer image were taken from American cancer society research center and DERMOFIT which are popular and widely focuses on skin cancer research. The classification system was supervised corresponding to the predefined classes of the type of skin cancer. Combining Self organizing map (SOM) and radial basis function (RBF) for recognition and diagnosis of skin cancer is by far better than KNN, Naïve Bayes and ANN classifier. It was also showed that the discrimination power of morphology and color features was better than texture features but when morphology, texture and color features were used together the classification accuracy was increased. The best classification accuracy (88%, 96.15% and 95.45% for Basal cell carcinoma, Melanoma and Squamous cell carcinoma respectively) were obtained using combining SOM and RBF. The overall classification accuracy was 93.15%.
Applying Deep Learning to Transform Breast Cancer DiagnosisCognizant
Deep convolutional neural networks can assist pathologists in breast cancer diagnosis by automatically filtering benign tissue biopsies, identifying malignant regions and labeling important cellular features like nuclei for further analysis. Automatic detection of diagnostically relevant regions-of-interest and nuclei segmentation reduces the pathologist’s workload, while ensuring that no critical region is overlooked, rendering breast cancer diagnosis more reliable, efficient and cost-effective.
Breast Cancer Detection from Mammography Images Using Machine Learning Algorithms (U-Net Segmentation and Dense Net Classifier implementation are in progress)
Machine Learning - Breast Cancer DiagnosisPramod Sharma
Machine learning is helping in making smart decisions faster. In this presentation measurements carried out on FNAC was analysed. The results were validated using 20 percent of the data. The data used for POC is from UCI Repository/
Mymmo is a digital mammogram analysis software developed by Travancore Analytics. It is a Computer Aided Detection (CAD) system that assists the diagnostician or radiologist in early detection of breast cancer.
Highly experienced engineers in the fields of medical imaging, 3D and 2D image processing and various databases have developed this highly unique and efficient diagnostic tool.
Skin Cancer Prognosis Based Pigment ProcessingCSCJournals
This paper develops a new computerized vision of skin cancer prognosis based on symmetry and color matching for lesion pigments. Initially, the lesion/tumor edge is detected and segmented. Then, the symmetrization is computed for all images to isolate benign (mole) tumor. The even symmetry parameter is introduced here to improve the symmetrization computations. The suspicious images would be nominated into one of three categories: melanoma, Basal Cell Carcinoma (BCC), or Squamous Cell Carcinoma (SCC) tumor depending on the symmetrization and pigment-color matching score table. Two matching procedures have been developed for nominating the suspicious images. First procedure matches pigment values with artificial spectrums of Reddish, Yellowish, Brownish, and Blackish. The second procedure matches pigment values with true malignancy/benign pigment database. The results of two procedures are compared over 40 pre-classified images. With Mean Squared Error (MSE) value equals to 0.003, procedure#1 satisfied 80% true classification while 92.5% for procedure#2. These results could be improved if lesion segmentation and/or spectrums/pigment-database are increased.
For the scope of this project, it was decided to analyze the data to form distinct clusters based on their tumor type. Unsupervised learning (K-means clustering and hierarchical clustering) were used. Also, it was decided to analyze this data as a classification task. Based on different attributes (primarily mass spectrometry analysis results for 12553 proteins) few classification algorithms were implemented to see if the model can generate the accurate label of cancer type.
Performance and Evaluation of Data Mining Techniques in Cancer DiagnosisIOSR Journals
Abstract: We analyze the breast Cancer data available from the WBC, WDBC from UCI machine learning with
the aim of developing accurate prediction models for breast cancer using data mining techniques. Data mining
has, for good reason, recently attracted a lot of attention, it is a new Technology, tackling new problem, with
great potential for valuable commercial and scientific discoveries. The experiments are conducted in WEKA.
Several data mining classification techniques were used on the proposed data. There are many classification
techniques in data mining such as Decision Tree, Rules NNge, Tree random forest, Random Tree, lazy IBK. The
aim of this paper is to investigate the performance of different classification techniques. The data breast cancer
data with a total 286 rows and 10 columns will be used to test and justify the different between the classification
methods and algorithm.
Keywords - Machine learning, data mining Weka, classification, breast cancer
Thermal Imaging: Opportunities and Challenges for Breast Cancer DetectionTarek Gaber
Thermal Imaging: Opportunities and Challenges for Breast Cancer Detection
Abstract:
Breast cancer is the most common cancer among women in the world. It is estimated that one in eight women, all over the wide, would develop breast cancer during her life. Breast cancer is considered one of the first-leading causes of cancer deaths among women. The early detection of breast cancer could save many women's life. Mammogram is one of the most imaging technology used for diagnosing breast cancer. Although mammogram has recorded a high detection and classification accuracy, it is difficult in imaging dense breast tissues, its performance is poor in younger women, it is harmful, and it couldn’t detect breast tumor that less than 2 mm. To overcome these limitations, it was found that there is a relation between the temperature and the presence of breast cancer. Utilizing this fact, infrared thermography could be a good source of breast images to study and detect cancer at the early stages which is crucial for cancer patients for increasing the rate of breast cancer survival.
This talk aims to give an overview of the thermal imaging technology, its possible applications in the medical field, focusing on its opportunities and challenges for the early detection of breast cancer and highlighting the state-of-the-art of this point.
Image processing techniques play a significant role in many areas in life, especially
in medical images, where they play a prominent role in diagnosing many diseases such
as detection of the brain tumor, breast cancer, kidney cancer, and the fractions.
Breast cancer is a common disease, regardless of the type of this disease, whether
it is benign or malignant, it is very dangerous and early detection may reduce the risk
of the disease spreading in the body leading to death. This work presents an approach
to detect breast cancer based on image processing algorithms, including image
preprocessing, enhancement, segmentation, Morphological operations, and feature
extraction to detect and extract the breast cancer region
A Comparative Study on the Methods Used for the Detection of Breast Cancerrahulmonikasharma
Among women in the world, the death caused by the Breast cancer has become the leading role. At an initial stage, the tumor in the breast is hard to detect. Manual attempt have proven to be time consuming and inefficient in many cases. Hence there is a need for efficient methods that diagnoses the cancerous cell without human involvement with high accuracy. Mammography is a special case of CT scan which adopts X-ray method with high resolution film. so that it can detect well the tumors in the breast. This paper describes the comparative study of the various data mining methods on the detection of the breast cancer by using image processing techniques.
Skin cancer is a term given to the uncontrolled
growth of strange skin cells. It occurs whenever unrepaired
DNA damages to skin cells trigger mutations, or any other
genetic defects, that lead the skin cells to multiply readily
and form malignant tumors. Image processing is a
commonly used method for skin cancer detection from the
appearance of the affected area on the skin. The input to the
system is that the skin lesion image so by applying novel
image process techniques, it analyses it to conclude about
the presence of skin cancer. The Lesion Image analysis tools
checks for the various Melanoma parameters Like
Asymmetry, Border, Colour, Diameter, (ABCD rule), etc. by
texture, size and form analysis for image segmentation and
have stages. The extracted feature parameters are
accustomed classify the image as traditional skin and
malignant melanoma cancerlesion.
Artificial Neural Network (ANN) is one of the
important branches of Artificial Intelligence, which has
been accepted as a brand-new technology in computer
science for image processing. Neural Networks is currently
the area of interest in medicine, particularly in the fields of
radiology, urology, cardiology, oncology, etc. Neural
Network plays a vital role in an exceedingly call network. It
has been used to analyze Melanoma parameters Like
Asymmetry, Border, Colour, Diameter, etc. which are
calculated using MATLAB from skin cancer images
intending to developing diagnostic algorithms that might
improve triage practices in the emergency department.
Using the ABCD rules for melanoma skin cancer, we use
ANN in the classification stage. Initially, we train the
network with known target values. The network is well
trained with 96.9% accuracy, and then the unknown values
are tested for the cancer classification. This classification
method proves to be more efficient for skin cancer
classification
Computer Vision for Skin Cancer Diagnosis and Recognition using RBF and SOMCSCJournals
Human skin is the largest organ in our body which provides protection against heat, light, infections and injury. It also stores water, fat, and vitamin. Cancer is the leading cause of death in economically developed countries and the second leading cause of death in developing countries. Skin cancer is the most commonly diagnosed type of cancer among men and women. Exposure to UV rays, modernize diets, smoking, alcohol and nicotine are the main cause. Cancer is increasingly recognized as a critical public health problem in Ethiopia. There are three type of skin cancer and they are recognized based on their own properties. In view of this, a digital image processing technique is proposed to recognize and predict the different types of skin cancers using digital image processing techniques. Sample skin cancer image were taken from American cancer society research center and DERMOFIT which are popular and widely focuses on skin cancer research. The classification system was supervised corresponding to the predefined classes of the type of skin cancer. Combining Self organizing map (SOM) and radial basis function (RBF) for recognition and diagnosis of skin cancer is by far better than KNN, Naïve Bayes and ANN classifier. It was also showed that the discrimination power of morphology and color features was better than texture features but when morphology, texture and color features were used together the classification accuracy was increased. The best classification accuracy (88%, 96.15% and 95.45% for Basal cell carcinoma, Melanoma and Squamous cell carcinoma respectively) were obtained using combining SOM and RBF. The overall classification accuracy was 93.15%.
Applying Deep Learning to Transform Breast Cancer DiagnosisCognizant
Deep convolutional neural networks can assist pathologists in breast cancer diagnosis by automatically filtering benign tissue biopsies, identifying malignant regions and labeling important cellular features like nuclei for further analysis. Automatic detection of diagnostically relevant regions-of-interest and nuclei segmentation reduces the pathologist’s workload, while ensuring that no critical region is overlooked, rendering breast cancer diagnosis more reliable, efficient and cost-effective.
Breast Cancer Detection from Mammography Images Using Machine Learning Algorithms (U-Net Segmentation and Dense Net Classifier implementation are in progress)
Machine Learning - Breast Cancer DiagnosisPramod Sharma
Machine learning is helping in making smart decisions faster. In this presentation measurements carried out on FNAC was analysed. The results were validated using 20 percent of the data. The data used for POC is from UCI Repository/
Mymmo is a digital mammogram analysis software developed by Travancore Analytics. It is a Computer Aided Detection (CAD) system that assists the diagnostician or radiologist in early detection of breast cancer.
Highly experienced engineers in the fields of medical imaging, 3D and 2D image processing and various databases have developed this highly unique and efficient diagnostic tool.
Skin Cancer Prognosis Based Pigment ProcessingCSCJournals
This paper develops a new computerized vision of skin cancer prognosis based on symmetry and color matching for lesion pigments. Initially, the lesion/tumor edge is detected and segmented. Then, the symmetrization is computed for all images to isolate benign (mole) tumor. The even symmetry parameter is introduced here to improve the symmetrization computations. The suspicious images would be nominated into one of three categories: melanoma, Basal Cell Carcinoma (BCC), or Squamous Cell Carcinoma (SCC) tumor depending on the symmetrization and pigment-color matching score table. Two matching procedures have been developed for nominating the suspicious images. First procedure matches pigment values with artificial spectrums of Reddish, Yellowish, Brownish, and Blackish. The second procedure matches pigment values with true malignancy/benign pigment database. The results of two procedures are compared over 40 pre-classified images. With Mean Squared Error (MSE) value equals to 0.003, procedure#1 satisfied 80% true classification while 92.5% for procedure#2. These results could be improved if lesion segmentation and/or spectrums/pigment-database are increased.
For the scope of this project, it was decided to analyze the data to form distinct clusters based on their tumor type. Unsupervised learning (K-means clustering and hierarchical clustering) were used. Also, it was decided to analyze this data as a classification task. Based on different attributes (primarily mass spectrometry analysis results for 12553 proteins) few classification algorithms were implemented to see if the model can generate the accurate label of cancer type.
Performance and Evaluation of Data Mining Techniques in Cancer DiagnosisIOSR Journals
Abstract: We analyze the breast Cancer data available from the WBC, WDBC from UCI machine learning with
the aim of developing accurate prediction models for breast cancer using data mining techniques. Data mining
has, for good reason, recently attracted a lot of attention, it is a new Technology, tackling new problem, with
great potential for valuable commercial and scientific discoveries. The experiments are conducted in WEKA.
Several data mining classification techniques were used on the proposed data. There are many classification
techniques in data mining such as Decision Tree, Rules NNge, Tree random forest, Random Tree, lazy IBK. The
aim of this paper is to investigate the performance of different classification techniques. The data breast cancer
data with a total 286 rows and 10 columns will be used to test and justify the different between the classification
methods and algorithm.
Keywords - Machine learning, data mining Weka, classification, breast cancer
Thermal Imaging: Opportunities and Challenges for Breast Cancer DetectionTarek Gaber
Thermal Imaging: Opportunities and Challenges for Breast Cancer Detection
Abstract:
Breast cancer is the most common cancer among women in the world. It is estimated that one in eight women, all over the wide, would develop breast cancer during her life. Breast cancer is considered one of the first-leading causes of cancer deaths among women. The early detection of breast cancer could save many women's life. Mammogram is one of the most imaging technology used for diagnosing breast cancer. Although mammogram has recorded a high detection and classification accuracy, it is difficult in imaging dense breast tissues, its performance is poor in younger women, it is harmful, and it couldn’t detect breast tumor that less than 2 mm. To overcome these limitations, it was found that there is a relation between the temperature and the presence of breast cancer. Utilizing this fact, infrared thermography could be a good source of breast images to study and detect cancer at the early stages which is crucial for cancer patients for increasing the rate of breast cancer survival.
This talk aims to give an overview of the thermal imaging technology, its possible applications in the medical field, focusing on its opportunities and challenges for the early detection of breast cancer and highlighting the state-of-the-art of this point.
Image processing techniques play a significant role in many areas in life, especially
in medical images, where they play a prominent role in diagnosing many diseases such
as detection of the brain tumor, breast cancer, kidney cancer, and the fractions.
Breast cancer is a common disease, regardless of the type of this disease, whether
it is benign or malignant, it is very dangerous and early detection may reduce the risk
of the disease spreading in the body leading to death. This work presents an approach
to detect breast cancer based on image processing algorithms, including image
preprocessing, enhancement, segmentation, Morphological operations, and feature
extraction to detect and extract the breast cancer region
A Comparative Study on the Methods Used for the Detection of Breast Cancerrahulmonikasharma
Among women in the world, the death caused by the Breast cancer has become the leading role. At an initial stage, the tumor in the breast is hard to detect. Manual attempt have proven to be time consuming and inefficient in many cases. Hence there is a need for efficient methods that diagnoses the cancerous cell without human involvement with high accuracy. Mammography is a special case of CT scan which adopts X-ray method with high resolution film. so that it can detect well the tumors in the breast. This paper describes the comparative study of the various data mining methods on the detection of the breast cancer by using image processing techniques.
Skin cancer is a term given to the uncontrolled
growth of strange skin cells. It occurs whenever unrepaired
DNA damages to skin cells trigger mutations, or any other
genetic defects, that lead the skin cells to multiply readily
and form malignant tumors. Image processing is a
commonly used method for skin cancer detection from the
appearance of the affected area on the skin. The input to the
system is that the skin lesion image so by applying novel
image process techniques, it analyses it to conclude about
the presence of skin cancer. The Lesion Image analysis tools
checks for the various Melanoma parameters Like
Asymmetry, Border, Colour, Diameter, (ABCD rule), etc. by
texture, size and form analysis for image segmentation and
have stages. The extracted feature parameters are
accustomed classify the image as traditional skin and
malignant melanoma cancerlesion.
Artificial Neural Network (ANN) is one of the
important branches of Artificial Intelligence, which has
been accepted as a brand-new technology in computer
science for image processing. Neural Networks is currently
the area of interest in medicine, particularly in the fields of
radiology, urology, cardiology, oncology, etc. Neural
Network plays a vital role in an exceedingly call network. It
has been used to analyze Melanoma parameters Like
Asymmetry, Border, Colour, Diameter, etc. which are
calculated using MATLAB from skin cancer images
intending to developing diagnostic algorithms that might
improve triage practices in the emergency department.
Using the ABCD rules for melanoma skin cancer, we use
ANN in the classification stage. Initially, we train the
network with known target values. The network is well
trained with 96.9% accuracy, and then the unknown values
are tested for the cancer classification. This classification
method proves to be more efficient for skin cancer
classification
Among the different types of skin cancer, melanoma is considered to be the deadliest and is difficult to treat at advanced stages. Detection of melanoma at earlier stages can lead to reduced mortality rates. Desktop-based computer-aided systems have been developed to assist dermatologists with early diagnosis. However, there is significant interest in developing portable, at-home melanoma diagnostic systems which can assess the risk of cancerous skin lesions. Here, we present a smartphone application that combines image capture capabilities with preprocessing and segmentation to extract the Asymmetry, Border irregularity, Color variegation, and Diameter (ABCD) features of a skin lesion. Using the feature sets, classification of malignancy is achieved through support vector machine classifiers. By using adaptive algorithms in the individual data-processing stages, our approach is made computationally light, user friendly, and reliable in discriminating melanoma cases from benign ones. Images of skin lesions are either captured with the smartphone camera or imported from public datasets. The entire process from image capture to classification runs on an Android smartphone equipped with a detachable 10x lens, and processes an image in less than a second. The overall performance metrics are evaluated on a public database of 200 images with Synthetic Minority Over-sampling Technique (SMOTE) (80% sensitivity, 90% specificity, 88% accuracy, and 0.85 area under curve (AUC)) and without SMOTE (55% sensitivity, 95% specificity, 90% accuracy, and 0.75 AUC). The evaluated performance metrics and computation times are comparable or better than previous methods. This all-inclusive smartphone application is designed to be easy-to-download and easy-to-navigate for the end user, which is imperative for the eventual democratization of such medical diagnostic systems.
Melanoma is a particularly dangerous type of skin cancer and is hard to treat in its later stages. Therefore, early detection is key in reducing mortality rates. In order to assist dermatologists in doing this, computer-aided systems have been designed for desktop computers. However, there is a desire for the development of mobile, at-home diagnostics for melanoma risk assessment. Here, we introduce a smartphone application that captures images and extracts ABCD features to classify skin lesions as either malignant or benign. The algorithms used are adaptive to make the process light and user-friendly, as well as reliable in diagnosis. Images can be taken with the phone's camera or imported from public datasets. The entire process of taking the image, performing preprocessing, segmentation and classification is completed on an Android smartphone in a short time. Our application is evaluated on a dataset of 200 images, and achieved either comparable or better performance metrics than other methods. Additionally, it is easy-to-download and easy-to-navigate for the user, which is important for the widespread use of such diagnostics.
Kalwa, U.; Legner, C.; Kong, T.; Pandey, S. Skin Cancer Diagnostics with an All-Inclusive Smartphone Application. Symmetry 2019, 11, 790. https://doi.org/10.3390/sym11060790
https://www.mdpi.com/2073-8994/11/6/790
SkinCure: An Innovative Smart Phone Based Application to Assist in Melanoma E...sipij
Melanoma spreads through metastasis, and therefore it has been proven to be very fatal. Statistical
evidence has revealed that the majority of deaths resulting from skin cancer are as a result of melanoma.
Further investigations have shown that the survival rates in patients depend on the stage of the infection;
early detection and intervention of melanoma implicates higher chances of cure. Clinical diagnosis and
prognosis of melanoma is challenging since the processes are prone to misdiagnosis and inaccuracies due
to doctors’ subjectivity. This paper proposes an innovative and fully functional smart-phone based
application to assist in melanoma early detection and prevention. The application has two major
components; the first component is a real-time alert to help users prevent skin burn caused by sunlight; a
novel equation to compute the time for skin to burn is thereby introduced. The second component is an
automated image analysis module which contains image acquisition, hair detection and exclusion, lesion
segmentation, feature extraction, and classification. The proposed system exploits PH2 Dermoscopy image
database from Pedro Hispano Hospital for development and testing purposes. The image database
contains a total of 200 dermoscopy images of lesions, including normal, atypical, and melanoma cases.
The experimental results show that the proposed system is efficient, achieving classification of the normal,
atypical and melanoma images with accuracy of 96.3%, 95.7% and 97.5%, respectively.
Dermatological diagnosis by mobile applicationjournalBEEI
Health care mobile application delivers the right information at the right time and place to benefit patient’s clinicians and managers to make correct and accurate decisions in health care fields, safer care and less waste, errors, delays and duplicated errors.Lots of people have knowledge a skin illness at some point of their life, For the reason that skin is the body's major organ and it is quite exposed, significantly increasing its hazard of starting to be diseased or ruined.This paper aims to detect skin disease by mobile app using android platform providing valid trustworthy and useful dermatological information on over 4 skin diseases such as acne, psoriasis content for each skin condition, skin rush and Melanoma. It will include name, image, description, symptoms, treatment and prevention with support multi languages English and Bahasa and Mandarin. the application has the ability to take and send video as well as normal and magnified photos to your dermatologist as an email attachment with comments on safe secure network, this app also has a built in protected privacy features to access to your photo and video dermatologists. The mobile application help in diagnose and treat their patients without an office visit teledermatology is recognized by all major insurance companies doctor.
Melanoma Skin Cancer Detection using Image Processing and Machine Learningijtsrd
Dermatological Diseases are one of the biggest medical issues in 21st century due to its highly complex and expensive diagnosis with difficulties and subjectivity of human interpretation. In cases of fatal diseases like Melanoma diagnosis in early stages play a vital role in determining the probability of getting cured. We believe that the application of automated methods will help in early diagnosis especially with the set of images with variety of diagnosis. Hence, in this article we present a completely automated system of dermatological disease recognition through lesion images, a machine intervention in contrast to conventional medical personnel based detection. Our model is designed into three phases compromising of data collection and augmentation, designing model and finally prediction. We have used multiple AI algorithms like Convolutional Neural Network and Support Vector Machine and amalgamated it with image processing tools to form a better structure, leading to higher accuracy of 85 . Vijayalakshmi M M ""Melanoma Skin Cancer Detection using Image Processing and Machine Learning"" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456-6470, Volume-3 | Issue-4 , June 2019, URL: https://www.ijtsrd.com/papers/ijtsrd23936.pdf
Paper URL: https://www.ijtsrd.com/engineering/other/23936/melanoma-skin-cancer-detection-using-image-processing-and-machine-learning/vijayalakshmi-m-m
Comparing the performance of linear regression versus deep learning on detect...journalBEEI
Melanoma is a type of deadly skin cancer. The survival rate of the patients can fall as low as 15.7% if the cancer cell has reached its final stage. Delayed treatment of melanoma can be attributed to its likeness to that of common nevus (moles). Two machine learning models were developed, each with a different approach and algorithm, to detect the presence of melanoma. Image classification is using the regression algorithm, and object detection is using deep learning. The two models are then compared, and the best model is determined according to the achieved metrics. The testing was conducted using 120 testing data and is made up of 60 positive data and 60 negative data. The testing result shows that object detection achieved 70% accuracy than image classification’s 68%. More importantly, linear regression’s 43% false-negative rate is noticeably high compared to convolutional neural network’s (CNN) 25%. A false-negative rate of 43% means almost half of sick patients tested using image classification will be diagnosed as healthy. This is dangerous as it can lead to delayed treatment and, ultimately, death. Thus it can be concluded that CNN is the best method in detecting the presence of melanoma.
Hybrid channel and spatial attention-UNet for skin lesion segmentationIAESIJAI
Melanoma is a type of skin cancer which has affected many lives globally. The American Cancer Society research has suggested that it a serious type of skin cancer and lead to mortality but it is almost 100% curable if it is detected and treated in its early stages. Currently automated computer vision-based schemes are widely adopted but these systems suffer from poor segmentation accuracy. To overcome these issue, deep learning (DL) has become the promising solution which performs extensive training for pattern learning and provide better classification accuracy. However, skin lesion segmentation is affected due to skin hair, unclear boundaries, pigmentation, and mole. To overcome this issue, we adopt UNet based deep learning scheme and incorporated attention mechanism which considers low level statistics and high-level statistics combined with feedback and skip connection module. This helps to obtain the robust features without neglecting the channel information. Further, we use channel attention, spatial attention modulation to achieve the final segmentation. The proposed DL based scheme is instigated on publically available dataset and experimental investigation shows that the proposed Hybrid Attention UNet approach achieves average performance as 0.9715, 0.9962, 0.9710.
A Survey On Melanoma: Skin Cancer Through Computerized DiagnosisChristo Ananth
Christo Ananth, M.Julie Therese, “A Survey On Melanoma: Skin Cancer Through Computerized Diagnosis”, International Journal of Advanced Research in Innovative Discoveries in Engineering and Applications[IJARIDEA], Volume 5, Issue 1, February 2020, pp: 9-18.
Segmentation and Classification of Skin Lesions Based on Texture FeaturesIJERA Editor
Skin cancer is the most common type of cancer and represents 50% all new cancers detected each year. The deadliest form of skin cancer is melanoma and its incidence has been rising at a rate of 3% per year. Due to the costs for dermatologists to monitor every patient, there is a need for an computerized system to evaluate a patient‘s risk of melanoma using images of their skin lesions captured using a standard digital camera. In Proposed method, a novel texture-based skin lesion segmentation algorithm is used and to classify the stages of skin cancer using probabilistic neural network. Probabilistic neural network will give better performance in this system to detect a lot of stages in skin lesion. To extract the characteristics from various skin lesions and its united features gives better classification with new approached probabilistic neural network. There are five different skin lesions commonly grouped as Actinic Keratosis (AK), Basal Cell Carcinoma (BCC), Melanocytic Nevus / Mole (ML), Squamous Cell Carcinoma (SCC), Seborrhoeic Keratosis (SK). The system will be used to classify the queried images automatically to decide the stages of abnormality. The lesion diagnosis system involves two stages of process such as training and classification. Feature selection is used in the classified framework that chooses the most relevant feature subsets at each node of the hierarchy. An automatic classifier will be used for classification based on learning with some training samples of each stage. The accuracy of the proposed neural scheme is higher in discriminating cancer and pre-malignant lesions from benign skin lesions, and it attains an total classification accuracy is high of skin lesions.
1. Machine Learning Engineer Nanodegree
Capstone Project: Detecting Melanoma with Machine Vision
Utilizing the Inception-v3 Convolutional Neural Network to Identify Melanoma in Moles
Ryan Ferrin
January 23, 2017
I. Definition
Project Overview
Skin cancer is the most common form of cancer in the United States.1
In fact, according to the American
Academy of Dermatology (AAD), 20% of Americans will develop skin cancer during their lifetime.
Furthermore, the AAD estimates that more than 8,500 people in the U.S. are diagnosed with skin cancer
every day.2
This amounts to approximately 5.5 million skin cancer diagnoses, in 3.4 million Americans
annually. The vast majority of these incidents are basal and squamous cell skin cancers rather than the
most dangerous form, melanoma, which accounts for slightly more than 75,000 cases annually.2
While melanoma accounts for a small amount, ~1%, of skin cancer cases, it results in over 70% of skin
cancer deaths. Of the approximately 13,500 annual deaths from skin cancer in the United States,
melanoma results in 10,000. Aside from the tragic loss of life due to skin cancer there is also an enormous
economic impact. It is estimated that the treatment of skin cancers in the U.S. costs $8.1 billion annually,
with melanoma accounting for $3.3 billion of that cost.3
While melanoma rates doubled in the U.S. from
1982 to 2011, there is reason to be optimistic. If detected early enough, almost all skin cancers, including
melanoma, can be cured if treated properly. In fact, the five-year survival rate of melanoma victims is
98% if it is detected and treated before it spreads to the lymph nodes. The survival rate decreases
drastically if the melanoma is detected and treated in later stages. 1
Ultimately, while skin cancer is the most common form of cancer in the United States, it is also easily
cured if detected and treated early enough. This means that early detection has the potential to save tens
of thousands of lives and billions of dollars in healthcare expenditures annually.
The recent advances in image recognition machine learning algorithms make the early detection of all
skin cancers an achievable goal. Skin cancer is primarily diagnosed visually, signs include: Changes in
size, shape or color of a mole or skin lesion, the appearance of a new growth on the skin, or a sore that
does not heal.4
All of these signs are visual features that can easily be identified, tracked, and monitored
by machine vision image recognition algorithms.
The ultimate goal of this project is to aid in the identification of skin cancer, specifically, melanoma, by
utilizing computer vision machine learning techniques. While it would be irresponsible, and likely illegal,
1
https://www.cdc.gov/cancer/skin/statistics/
2
https://www.aad.org/media/stats/conditions/skin-cancer
3
http://www.cancer.org/cancer/cancercauses/sunanduvexposure/skin-cancer-facts
4
https://www.aad.org/media/stats/conditions/skin-cancer
2. to claim that this tool is a diagnostic tool, its purpose is to assist with early detection, not replace the
advice from a licensed medical professional.
This project will be focused on detecting melanoma in moles utilizing the Inveption-v3 image recognition
algorithm. This algorithm will be trained on open-source pre-labeled images of both benign and
malignant (melanoma) moles. Upon completion of the model, an image of any mole, ideally taken from a
smartphone app, can be evaluated by the model, and a probability will be provided for whether or not the
mole is malignant or benign.
Problem Statement
The ultimate goal of this project is to develop an image recognition model that can identify whether a
mole is benign or malignant (with melanoma). Such a model would allow people to use a device such as a
smartphone, in the comfort of their homes, to aid in the detection of melanoma by simply uploading a
photo for analysis. Skin cancer is the most common form of cancer in the United States. Therefore, this
model has the potential to save thousands of lives and billions of dollars annually.
The project strategy is as follows:
1. Obtain and pre-process open-source labeled photos of benign and malignant moles.
2. Retrain the Incpetion-v3 model to create a model for analyzing these images.
3. Evaluate the model performance.
4. Adjust the model to improve detection accuracy.
While achieving an extremely high level of accuracy is unlikely at this point, the goal is to achieve above
human-level diagnostic accuracy, the determination for this accuracy level is discussed in greater detail in
the Benchmark section.
Metrics
For the metrics used in this project it is necessary to understand some the performance measures used
when evaluating a model. There are four such measures which will be used to calculate the metrics. See
Figure 1.
tp: True Positive
tn: True Negative
fp: False Positive
fn: False Negative
Figure 1. The four measures are as follows. True positive: When melanoma is predicted and it the actual
diagnosis is melanoma. False positive: When melanoma is predicted but the actual diagnosis is benign.
False negative: When benign is predicted but diagnosis is melanoma. True negative: When benign is
predicted and when diagnosis is benign.
3. Accuracy:
For many models accuracy is a valuable metric to determine the model’s predictive power. However, for
reasons described below, it will only be used for informative purposes and not legitimate model
evaluation.
Precision: 5
For this project precision will be the fraction of accurate melanoma predictions out of all the actual
melanoma diagnoses.
Recall:6
Recall is the fraction of melanoma cases that were predicted out of all of the melanoma cases.
F1 Score:7
The F1 score is used to account for both precision and recall. The F1 score is essentially a weighted
average of precision and recall. It allows for a view of how accurate the model is with respect to the two
error types discussed above.
5
https://en.wikipedia.org/wiki/Precision_and_recall
6
https://en.wikipedia.org/wiki/Precision_and_recall
7
https://en.wikipedia.org/wiki/F1_score
4. Primary Model Evaluation Metric: Recall
Because the purpose of this project is to diagnose melanoma, there are several factors that need to be
taken into consideration. The first is the potential errors that can be made by the model. These errors are
known as type I and type II errors.8
A type I error is associated with false positives; in this project it is
when melanoma is diagnosed when the mole is actually benign. This is measured by precision; the greater
the number of type I errors the lower the precision. Type II errors are associated with false negatives. In
this project it is when a mole is diagnosed as benign but actually has melanoma. These errors are
measured by recall; the more type II errors the lower the recall will be and vice versa.
With respect to diagnosing diseases, generally both types of error can lead to significant concerns. In the
event that a treatment is dangerous and costly, it can be of great concern to make a type I error and
recommend treatment when it is not necessary. If the disease is dangerous and requires treatment, it is
also of great concern if a type II error is made and treatment is not recommended when it should be.
In the case of mole biopsy/excision, the treatment is not costly and it is relatively safe. Therefore, type I
errors are not generally an issue. Whereas, if melanoma is not detected, it can be life-threatening. This
makes type II errors far more dangerous in skin cancer diagnosis. Due to this fact, recall (also known as
sensitivity) will be used as the primary metric for measuring the models accuracy and comparing it to the
benchmark discussed in the Benchmark section.
II. Analysis
Data Exploration
The dataset used for this project is a collection of 2,000 mole images. The images were obtained from the
International Society for Digital Imaging of the Skin’s website. They were curated for the International
Skin Imaging Collaboration: Melanoma Project (ISIC). The goal of ISIC is to develop standards for
dermatological imaging. To further this goal, ISIC has developed an open source public access archive of
skin images which can be used for testing and developing automated diagnostic systems.9
The 2000
images were prepared from the archive of more than 13,000 images.10
The dataset consists of 2,000 mole images. The composition of the dataset is represented in Figure 2.
Diagnosis # of Images % of Total
Benign 1372 68.6%
Melanoma 374 18.7%
Seborrheic Keratosis (Benign) 254 12.7%
Figure 2. Image dataset composition. This is the number of images and the percentage of the total dataset
for the benign, melanoma, and seborrheic keratosis (benign) categories.
Benign: indicates that there is no cancer or other abnormalities present.
8
https://en.wikipedia.org/wiki/Type_I_and_type_II_errors
9
http://isdis.net/isic-project/
10
https://challenge.kitware.com/#challenge/n/ISIC_2017%3A_Skin_Lesion_Analysis_Towards_Melanoma_Detection
5. Melanoma: Indicates that the mole is malignant with melanoma.
Seborrheic Keratosis (Benign): Indicates that the mole is not cancerous but does have an
abnormality known as Seborrheic Keratosis.
All of the images have been submitted to ISIC by leading clinical centers and have been labeled
according to a professional diagnosis. In total, there are 2000 partially standardized images. The majority
of the images (70%) are benign moles, with melanoma accounting for about 20%, and Seborrheic
Keratosis (SK) accounting for about 13%.
The images are partially standardized with respect to the formatting and photographic techniques.
However, there are some images that lack consistency and have variations in formatting (Figure 3 and
Figure 4 below).
Benign Melanoma Seborrheic Keratosis (Benign)
Figure 3. Partially standardized images. The majority of images in the dataset are formatted in this manner.
The mole is centered in the image with no other objects or markings present.
Figure 4. Abnormal images found in the dataset. A portion of the images contain nonstandard elements.
Above from left to right: the image has markings surrounding the mole. The image has a black ring
surrounding it as a result of the imaging device. The image has a circular sticker located near the mole.
The reason behind some inconsistencies in standardization is that a wide variety of different institutions
have contributed the images to the ISIC archive. Many of these institutions have unique imaging practices
that do not necessarily reflect the ISIC practices. These abnormalities may present complications into the
image recognition algorithm. For example, the black ring or the sticker in the images may be identified as
a feature by the algorithm. This may be problematic because it does not necessary reflect any type of
diagnosis however, it may negatively influence the algorithm to think that it has relevancy that does not
exist.
Exploratory Visualization
Sticker
Markings
Black Ring
6. Because the image data was collected from a variety of sources it was necessary to examine the quality of
the data and determine if it was consistent. This evaluation was conducted based on the abnormalities
within the dataset that were discussed above.
There are three major forms of abnormality, the first is a circular sticker in the image. This is a small
sticker located next to the mole. The second is markings drawn near or surrounding the mole. The third is
a black circular ring surrounding the image These abnormalities may adversely affect the image
recognition algorithm. This is because the abnormality may be identified as a feature of one of the
diagnosis types, and subsequently impact how the algorithm labels images that are processed through it.
Because of this fact it was necessary to analyze all of the pictures and identify whether the abnormalities
were associated with any particular diagnosis. If the abnormalities are randomly distributed throughout
the images then this would not be a concern, because it would have an equal impact on all of the classes.
However, if it is determined that the abnormality is associated with a specific class, then all of the images
with that abnormality will have to be removed to prevent bias from developing in the model. It is crucial
to analyze this characteristic of the data as the entire model could be compromised if it is not resolved.
Figure 5 displays the results of the analysis. The graph displays the type of abnormality, measured by
count. With the proportion of each diagnosis associated with the abnormality identifies by color.
Figure 5. Abnormalities displayed by diagnosis type. From the figure it is evident that bias would be
introduced into the model if some images are not removed, specifically the images with stickers in them.
It is clear that the abnormalities are not distributed evenly throughout the classes. Some images will have
to be removed to prevent the introduction of bias into the model. The sticker abnormality is only present
in benign images. This indicates that the sticker would most certainly be recognized as a feature of being
benign. Of course this not actually the case and it will not generalize to unseen data. The images with
stickers were likely all contributed from one institution, so the stickers are an artifact of their imaging
techniques and of course not an actual indication of being benign. When images that did not come from
this institution are tested, the sticker feature will not generalize to the new images. There is a similar
situation for the circular ring abnormality, there is no SK present with this abnormality. The marker seems
to be representative of the actual proportions of diagnoses in the dataset and therefore does not need to be
addressed. Ultimately, there are abnormalities in the data that will need to be addressed prior to
generating the model
0
50
100
150
200
250
300
Total Sticker Markings Black Ring Other
#ofImages
Diagnosis
Type of abnormal image by diagnosis type
Benign Melanoma Seborrheic Keratosis (Benign)
7. Algorithms and Techniques
The analysis will be conducted using a convolutional neural network(CNN). A CNN is a machine
learning technique that loosely emulates the way in which the human brain functions. A CNN takes
inputs, and through its connections makes a prediction of what class the input belongs to. Based on the
accuracy of the prediction, the connections will be updated to improve the prediction in the future. CNNs
consists of a network with units (neurons) which have learnable weights and biases. Each neuron
receives inputs, for this project the inputs are raw image pixels. Convulsions then compute the outputs of
the neurons which is generated by performing a dot product based on the neuron’s weight and the region
they are connected to. The neurons are arranged in layers’ referred to as “hidden layers”. The
calculations performed on the input in the hidden layers can then be sent through additional layers such as
“pooling layers” examples are MaxPool and AvgPool (Figure 6), which are used to reduce the spatial
size of the representation which reduces the amount of computation and parameters in the network. Also
fully connected layers can contain neurons connected to all the neurons in the previous layer. A loss
layer such as Softmax (Figure 6) penalizes the model based on the deviation between the predicted and
true labels. This information is then sent via a backpropagation algorithm to update the neuron weights
based on the error made in the prediction. The whole network expresses a single differentiable score
function. This score can then be used to improve the model by learning which inputs map to which class
through this iterative process. 11
CNNs perform extraordinarily well for image recognition tasks and are the basis of most state-of-the-art
computer vision models.12
The goal of using a CNN is that it builds many layers of feature detectors
which can account for the spatial arrangement of pixels in an image. In the context of neural networks,
convolutional neural networks can be thought of as using many identical copies of the same neuron. This
provides the benefit of a network that has an extensive number of neurons, which express a
computationally large model, while limiting the number of parameters (the values which describe how the
neurons behave) to a fairly small amount. This makes it easier for the CNN to learn the model and
reduces errors.13
The specific CNN algorithm that will be used is known as the Inception-v3 algorithm. Inception-v3 was
selected for several reasons. First, upon its release in December of 2015, it demonstrated substantial gains
over the state-of-the-art computer vision algorithms. Second, it is developed for computational efficiency
and low parameter count which enables its use in a wider range of applications such as big data and
mobile vision. Finally, through the TensorFlow library, Google has released the capability to retrain just
the final layer of the Inception-v3 algorithm through a process known as transfer learning. The Incpetion-
v3 architecture can be seen in Figure 6.14
Without the ability to retrain just the final layer of the algorithm its use would not be feasible for this
project. It would require weeks of constant processing to train the model because it has millions of
parameters. Ultimately, the process of transfer learning allows for the use of a trained Inception-v3 model,
which retains all of its feature weights, and applies them to the new classes (images of moles) through a
retraining process. This allows for the use of one of the best models available, with a personal computer
and a relatively short amount of time.
11
http://cs231n.github.io/convolutional-networks/#fc
12
Szegedy, C., Vanhoucke, V., & E. (2015, December 02). Rethinking the Inception Architecture for Computer
Vision. Retrieved January 01, 2017, from https://arxiv.org/abs/1512.00567
13
http://colah.github.io/posts/2014-07-Conv-Nets-Modular/
14
https://arxiv.org/pdf/1310.1531v1.pdf
8. Figure 615
. The fully trained Inception-v3 model has millions of parameters and has been trained on millions of
images in the Imagenet image database.16
When retraining, the model retains what it learned about the Imagenet
images and applies that information to the new images. This technique saves weeks of processing time.
The Inception-v3 model is available through the Tensorflow library. It has a variety of customizable
parameters to allow for improving the model. The tunable hyper-parameters relevant to this project are
listed below.
1. how_many_training_steps: How many training steps to run before ending
2. learning_rate: How large a learning rate to use when training
3. testing_percentage: What percentage of images to use as a test set
4. validation_percentage: What percentage of images to use as a validation set
5. eval_step_interval: How often to evaluate the training results
6. train_batch_size: How many images to train on at a time
7. test_batch_size: How many images to test on
8. validation_batch_size: How many images to use in an evaluation batch
9. flip_left_right: Whether to randomly flip half of the training images horizontally
10. random_crop: Determines how much of a margin to randomly crop off the training images
11. random_scale: Determines how much to randomly scale up the size of the training images
12. random_brightness: Percentage determining how much to randomly multiply the training image
input pixels up or down by
15
https://research.googleblog.com/2016/03/train-your-own-image-classifier-with.html
16
http://image-net.org/about-overview
Schematic Diagram of Inception-v3
Final layer trained
on new images
Layers that have already been trained on the Imagenet dataset
Input:
Image
Output:
Class Label:
(benign or malignant)
9. Benchmark
A typical assessment of a machine learning model’s effectiveness is whether or not it is better than a
human performing the same task. In the case of a model focused on diagnosing skin cancer this is a
logical strategy. In order for the model to be valuable it likely needs to be at least as effective as the
average human physician.
According to a study conducted by Rolfe17
, which analyzed the accuracy of 6546 skin biopsies/excisions,
physicians had a diagnostic sensitivity (recall) of 76% with respect to melanoma diagnoses. As discussed
in the Metrics section, recall will be used as the primary evaluation metric. Therefore, the benchmark for
this model is:
Recall > 76%
III. Methodology
Data Preprocessing
Step 1: Remove unnecessary items from the dataset
For the ISIC dataset superpixel images for all of the mole images were provided. Since these will not be
necessary for generating this model, they all had to be removed (Figure 7).
Step 2: Remove abnormal pictures
As discussed in the Exploratory Visualization section, there were abnormal pictures that could potentially
bias and compromise the model. These images were all identified and removed. This amounted to 229
images being removed from the dataset. (Figure 8).
Step 3: Divide the images into three separate folders based on their diagnoses. Benign, Melanoma
and Seborrheic Keratosis.
The implementation of the Inception-v3 model being used analyzes images within folders and classifies
each image as the name of the folder. Therefore, all of the images for each specific diagnosis need to be
within the same folder.
Step 4: Remove the Seborrheic Keratosis folder from the analysis
Because the goal of this project is to determine whether a mole is benign (without any abnormality) or
malignant (with melanoma, the Seborrheic images will add unnecessary complexity to the model.
Step 5: Reduce the number of images in each image directory
After removing the superpixel images, the dataset of 2000 mole images is ~8GB. While more data is
almost always superior to less data for machine vision tasks, it was not feasible to work with such a large
dataset due to limited computational capabilities and need to transfer the files for submission.
300 training images and 30 test images, were randomly selected from the dataset for both of the classes,
660 total. The final file size is ~2GB. This allows for more manageable model building and evaluation.
17
https://www.ncbi.nlm.nih.gov/pubmed/22571558
10. Step 6: Split images into training, testing and validation sets
When the model is initialized it analyzes the folders in the image directory and splits the images into
training, testing and validation sets prior to analyzing them. 10% are divided to test, 10% to validation
and 80% to training. After these preprocessing steps have completed the analysis can begin.
Figure 7 (left). An example superpixel image from the dataset. While superpixel images can be valuable
for other analyses, they are not necessary for this project, therefore, they were removed.
Figure 8 (right). A collection of abnormal images that had to be removed from the dataset in order to
prevent bias. In this example, all of the stickers were found in benign images.
Implementation
Upon completion of the Tensorflow setup (see Read Me file), the model can be initiated using the
example retrain.py file. This python file contains all of the necessary components and calculations to run
the algorithm and analyze images. Alterations to the retrain.py file were necessary to meet the goals of
this project. These steps are as follows:
Step 1: Calculate true positive, false positive, false negative and true negative measures
The model only calculates accuracy by default. Because the goal is to measure the model’s performance
using recall, it was necessary to calculate the tp, fp, fn, tn, measures so that they could be utilized to calculate
recall.
Step 2: Display the hyper-parameters being adjusted each time the model is ran
Because the goal is to optimize the model with the highest recall possible, it is useful to have all of the
hyper-parameters being used and with their respective values. The model does not do this by default so it
was necessary to develop and implement this functionality.
Step 3: Display the measures: true positive, false positive, false negative and true negative
It was also necessary to display the measures in order to help model optimization.
Superpixel Image Abnormal Images Removed from Dataset
11. Step 4: Calculate & display model metrics performance: precision, recall, f1 and accuracy
With respect to overall model performance the only metric provided by default is accuracy. As mentioned
previously, accuracy is not a useful metric to use with this project because it will be skewed by true
negatives. Therefore, calculations were implemented for precision, recall, f1, and accuracy (in order to
verify consistency with the model’s accuracy)
Step 5: Append all performance metrics and hyper-parameters to a CSV file for analysis
Because it is necessary to run many trials with many variants of hyper-parameter settings to optimize the
model, all the metrics and hyper-parameter information is appended to a CSV file which can be analyzed
for optimal parameter tuning.
Running the Incpetion-v3 model.
After all of the necessary installation and setup steps are complete the process for analyzing images with
the model is relatively simple. The steps below outline how the process/model function.
1. Navigate to the tensorflow directory.
2. Run the appropriate code directed to the images that will be analyzed.
3. The retrain.py file will be called and begin the analysis on the selected images.
4. The latest version of the inception-v3 model will be downloaded
5. The model calculates “bottleneck” values for each image, these are the values the model uses to
classify the image.
6. The images are randomly divided into training, testing and validation sets.
7. The model begins training: the model predicts which class the image belongs to, compares the
prediction to the actual label, and updates the final layer’s weight using the backpropagation
process. The validation images are used during this step (Figure 9).
8. Finally, the test images, those that did not contribute to the model training, are analyzed. The
predictions made for these images determine the overall model accuracy.
9. The model displays the results including all of the customized metrics and values (Figure 10).
Figure 9 (above). The Inception-v3 model displays output for three
of the training processes. First, train accuracy shows what percent
of images in the current batch were predicted correctly. Second, cross
entropy is a loss function that demonstrates how well the learning process
is progressing because the objective of training is to minimize the loss function.
Finally, the validation accuracy represents how well the model predicts randomly-
selected images from different batches.
Figure 10 (left). Output after training is complete. This output was
obtained from running the model with all default values for the hyper-parameters,
except test_batch_size which was manually set at 10% of the dataset (60).
Top, the overall accuracy of the model. Middle, the hyper-parameters and their values.
Bottom, the metrics demonstrating the model’s performance.
Recall is 67.9%, the average of three trials with default values was 70.9%. This leaves
much room to improve the model to achieve a performance greater than the 76%
benchmark.
12. Refinement
After the default model obtained a recall performance of 70.9%, it was necessary to adjust some of the
hyper-parameters in order to increase the performance to achieve higher than the benchmark of 76%.
The steps for identifying a better performing model were:
1. Adjusted the parameters, beginning with how_many_training_steps holding all other parameters
constant with default values.
2. Began with 1000 training steps and increased until it was evident that recall performance would
not improve with additional training steps.
3. Ran the model three times with each number of training steps to get an average trial performance.
The average of three runs for 2000 steps was the highest performing.
4. Selected 2000 training steps for the model. While the average performance of 2000 training steps
surpassed 76% accuracy with ~80%, it was still advantageous to continue to search for a more
optimized model to make it more robust.
5. 2000 training steps was then made the default amount and the optimal learning_rate was then
searched for in a similar manner. Beginning with the default of .01 and increasing.
6. The default value of .01 performed the best so it was selected for the model.
The limitations to this approach are readily apparent. There are only a small number of variations being
tested. However, there was a tradeoff between improved performance and computing time. To run just
these analyzes it took ~5 hours on the computer being used. This computational speed motivated not
checking learning rates less than the default. Because the model already performed superior to the
benchmark, adjustments that decreased the speed of the model were not evaluated.
Figure 11 (left). 2000 training steps performed the best. (right) The default learning rate of .01 performed
the best.
65.3%
79.7%
77.0%
70.9%
69.0%
60%
65%
70%
75%
80%
85%
0 1000 2000 3000 4000 5000 6000
Averagerecallofthreetrials
Number of training steps
Average recall based on number of training steps
79.7% 79.5%
71.6%
66.8%
76.4%
73.6%
66%
68%
70%
72%
74%
76%
78%
80%
82%
0 0.05 0.1 0.15 0.2
Averagerecallofthreetrials
Learning rate
Average recall based on learning rate
13. IV. Results
Model Evaluation and Validation
As discussed a process of trial and error was conducted to generate a model that consistently performs
superior to humans at diagnosing melanoma. The only hyper-parameter that required adjustment to
overcome the benchmark of 76% recall was the number of training steps. However, training steps alone
did not make the model sufficiently robust, so learning rate was also adjusted until the model consistently
performed above 76% recall.
In order to evaluate the models’ robustness, 60 new images (10% the size of the dataset) were
incorporated into the dataset. If the model was overfitting to the data, introducing these new images
should have had an effect on the performance. This is exactly what was seen. After introducing the new
images, the average recall of the model dropped to about 72%. With the new images, further model tuning
may be necessary.
That being said, the model’s results probably should not be trusted for a variety of reasons. First, the
dataset is very small and it was collected in an uncontrolled manner (it was self-submitted) so there are
possibly biases in the data, such as lighting, camera focus, angles, camera distance from mole, etc. The
model may be detecting these subtle differences in photographic technique rather than true features of
melanoma. While it would be fantastic if a diagnostic model superior to human physicians could be
generated from 600 images, this simply does not seem plausible. These results should probably not be
taken for true diagnostic accuracy, but rather features of the data and the limited number of images.
Justification
Ultimately, the final solution does provide results that are consistently superior to the benchmark.
However, for the reasons discussed above these results should probably not be trusted. Performing a
legitimate statistical test for significance is not feasible on these results due to the small sample size of
trials and the methodological differences between this project and the research paper from which the
benchmark was generated. To conduct a legitimate test for statistical significance, such as a t-test, many
more analyses would be needed, which would take a prohibitively large amount of time with the current
computing resources. The model demonstrates promise and the potential for a sophisticated diagnostic
model in the future. But there just simply is not enough evidence to demonstrate that the model generated
in this project is superior to human diagnostic abilities.
V. Conclusion
Free-Form Visualization
After the Inception-v3 model is completely trained, it can evaluate new images and provide a probability
that the image belongs to each of the classes in the training dataset, for this project that is whether the
mole in the image has melanoma or is benign. In Figure 12 three images have been evaluated by the
model. Two of the images are moles located on my stomach, both of which are almost certainly benign.
The third is a scar that formed after a mole was excised. This scar is located near the other moles being
analyzed on my stomach.
14. Figure 12. Images of moles (left two) and a scar from a mole removal (right) were analyzed to determine
the probability that the moles in the images are benign or have melanoma. The model has given the first
mole ~76% probability of being benign. The second mole image was given a 65% chance of having
melanoma. The third image was given a 96% of being benign. There is no reason to believe the mole in
the second image actually has melanoma, but the model seems to be accurate for the other two images.
Reflection
Because I have a large number of moles, I have been interested in this concept for quite a long time. I feel
that a machine learning algorithm could be far superior to any human if given enough training data. So,
when given the opportunity to select a project I immediately knew this was going to be my choice.
I began the project by ensuring that I could obtain images of moles. It took quite a long time to find the
images that I used. It also took a significant amount of time to obtain them after I found them due to the
file size and difficulty navigating the website that contains them.
I then began searching for a powerful machine vision algorithm that I could utilize. I considered
developing an algorithm from scratch. I quickly realized this would be unwise because there are many
models that have been developed extensively which are freely available. I began with scikit-image. I had
limited success developing some rudimentary models from the preexisting scikit-image models. I then
learned that TensorFlow was open-source, and it was developed by Google. I immediately knew that I
wanted to take advantage of this opportunity. I quickly found tutorials for using TensorFlow and
retraining Inception-v3.
Although tutorials are available, they are very basic and omit many of the details that I needed to
customize the model, so initially it was very difficult for me to use TensorFlow and I spent a significant
amount of time learning how to use it. Almost immediately I realized I would have to install a Linux
virtual machine on my computer because much of the software dependencies for TensorFlow either have
not been developed for Windows or do not work well for Windows at this time.
Benign moles (almost certainly) Scar from removed benign mole
15. After countless hours of learning how to use Tensorflow, I finally was able to get it to function. I then
cleaned the image data, randomized and separated it into smaller classes for the analysis. I then began
analyzing the data and adjusting the hyper-parameters to optimize the model. The Inception-v3 model is
an extraordinarily effective model and it was great getting the opportunity to utilize it for this project. I
still feel slightly skeptical of the great results I obtained for only having 600 images in the dataset,
however it definitely is cause for optimism with respect to machine vision capabilities.
Improvement
There are two fundamental limitations to the model which, if overcome, would likely improve it
substantially. The first is a lack of computing resources. The second is lack of data.
Need for more computational resources
Although the retrained model only requires a fraction of the time to use when compared to training from
scratch, it still requires a substantial amount of time and computational resources. Testing different
variations of hyper-parameters requires an extensive amount of time. Also, some of the distortions simply
could not be used because the time necessary was prohibitive. The model is equipped with four
distortions which can be used to randomly distort the photos, which can help the model generalize to
unseen data more effectively.
The distortions are:
1. flip_left_right
2. random_crop
3. random_scale
4. random_brightness
These parameters are described in more detail in the Algorithms and Techniques section. All of the
parameters manipulate the photos in some manner, which can help remove potential bias from an image
class. Unfortunately, when attempting to utilize these distortions the computer being used was not
powerful enough to make it practical. It took several minutes just to analyze one batch, as opposed to
several batches per second when the distortions were not used. That being said, these distortions could
potentially improve the model. It just was not practical to use them for this project.
Lack of image data
For developing an effective machine vision model, 600 images is likely far too few. Of course, the
number of images was limited in an effort to reduce evaluation time and make transferring files simpler.
However, even the total 13,000 images available on the ISIC website is likely far too small. For
something as nuanced as mole features, it will possibly take millions of labeled images to develop a truly
effective human-level model. More images would almost certainly improve the model even if nothing
else was adjusted. Ultimately, there are ways in which this model could be improved greatly. I believe
that in the not too distant future there will be machine learning algorithms that are superior to humans
with respect to diagnosing melanoma.