Skin-Net: a novel deep residual network for skin lesions classification using multilevel feature extraction and cross-channel correlation with detection of outlier

Abstract

Human Skin cancer is commonly detected visually through clinical screening followed by a dermoscopic examination. However, automated skin lesion classification remains challenging due to the visual similarities between benign and melanoma lesions. In this work, the authors proposed a new Artificial Intelligence-Based method to classify skin lesions. In this method, we used Residual Deep Convolution Neural Network. We implemented several convolution filters for multi-layer feature extraction and cross-channel correlation by sliding dot product filters instead of sliding filters along the horizontal axis. The proposed method overcomes the imbalanced dataset problem by converting the dataset from image and label to vector of image and weight. The proposed method is tested and evaluated using the challenging datasets ISIC-2019 & ISIC-2020. It outperformed the existing deep convolutional networks in the multiclass classification of skin lesions.

Graphical Abstract

Introduction

Artificial intelligence and the Internet of Things (IoT) in health care have become an urgent necessity in recent years due to the smart cities and the COVID-19 pandemic, as discussed in [1, 2]. The word “cancer” is a term that refers to a group of disorders caused by abnormal cell proliferation. Cancer invades or spreads to other body parts due to abnormal cell proliferation [3]. In general, Cancer-related deaths have risen dramatically. Recently, Skin cancer has been a leading cause of mortality, especially in regions with high solar activity [4].

Detecting melanoma among benign skin lesions is still a big issue. There are different kinds of skin lesions, such as benign (nevus), cancerous (melanoma), basal cell carcinoma (BCC), pigmented benign keratosis (BKL), and squamous cell carcinoma (SCC) [5]. A visual inspection is required because diverse skin lesions are similar [6]. The naked eye is challenging and requires an expert dermatologist with high skill. Instead of visual examination by the naked eyes directly of the human skin, Dermoscopy, a noninvasive imaging technology, enhances melanoma diagnosis. But in typical clinical circumstances, the dermatologist's ability to identify melanoma using dermoscopic images only reached less than 80% accuracy [7]. As a result, researchers have concentrated on recognizing melanoma to assist clinicians in distinguishing between melanoma and benign tumors early to preserve the patient's life [8].

For image classification, there are two main approaches. The first approach is based on extracting handcrafted features from images. In the second approach, Deep Convolutional Neural Networks (DCNN) can learn from features hierarchically. In [9], Ravi and his colleagues summarize the advantages of DCNN over the first approach in medical image classification. There are two main challenges to developing computer-aided diagnosis systems. These challenges are the lack of data and the algorithms used for image processing [10,11,12].

For any early skin cancer diagnosis and therapy, efficient diagnosis is essential. Since Convolutional Neural Networks (CNNs)-based approaches significantly increase prediction accuracy, several researchers have been concentrating on them recently [13]. Due to its autonomous feature design and self-learning capabilities, Deep Learning (DL) based algorithms are being heavily studied for different tasks such as diagnosing cardiovascular events [14] and skin cancer. With deep neural networks, high performance can be attained at the expense of widening, deepening, and increasing the resolution of the CNN, which forces the design to include extra parameters and requires high processing resources for training and testing.

Skin cancer classification is generally difficult because of artifacts, disparities in image resolution, and weaker discriminating features across different forms of cancer. Patients find clinical procedures for skin lesions [15, 16] complicated and uncomfortable, and they are ineffective in distinguishing between different lesion types. Machine Learning and Computer Vision methods are promising tools to overcome the different challenges of skin lesion classification. The initial stage in CAD systems is to produce features for classifying normal and pathological lesions. The CAD systems are helpful in the early diagnosis of skin cancer and, consequently, reduce the mortality rate.

Due to the artifacts, the difference in image resolution, and the high similarity between different lesions, skin cancer classification is still challenging. All these issues motivated the authors to design a new RDCNN called Skin-Net. We used this architecture to develop a reliable diagnostic technique with a high classification rate for detecting melanoma on the skin in its early stages. Thus, the proposed method is an excellent tool for finding and screening lesions for faster treatment and a better chance of survival. The following are the primary contributions and benefits of this work:

• 1. A new end-to-end trained residual deep neural network is proposed.

• 2. We designed full and robust automated skin lesions classified using a classification system. with high classification rates.

• 3. Multiple convolution filters are applied to the same input in the proposed architecture.

• 4. Multilevel feature extraction using a 3 × 3 and 1 × 1 local concatenated filter.

• 5. Several features from multiple filters are used to increase the effectiveness of skin lesion categorization.

• 6. We performed a cross-channel correlation instead of convolution on both a spatial and a channel-wise domain. We ignored the spatial dimensions using a 1 × 1 convolution through the residual block.

• 7. Unlike shallow networks, the proposed Skin-Net does not produce high training errors.

• 8. A new algorithm to serve the class imbalance.

• 9. Challenging datasets were utilized to evaluate the proposed method.

This paper is structured as follows: “Literature review”, an overview of the state-of-the-art. “Proposed method” describes the proposed method. The performed experiments, the obtained results, and a comparison with the existing models are presented in “Experimental results and discussion”. Finally, the conclusion is presented in “Conclusion”.

Literature review

Several efforts have been carried out in recent decades to assist physicians in identifying skin lesions appropriately. An emerging field of study is computer-aided systems for skin lesion diagnostics. As discussed earlier, there are two approaches to image classification: traditional machine learning and deep learning.

Traditional machine learning

The remarkable papers [18, 19] review the accuracy of various CAD systems. The ABCD and 7-point checklist rules are widely used in classifying skin images based on asymmetry, border, color, and differential structures (shapes). But these methods are inaccurate with low-performance measures [17].

A probabilistic neural network (PNN) is used in a CAD system to classify skin lesions, as has been suggested by Kostopoulos et al. [20]. This method achieved 76.2% as a classification rate. Ozkan and Koklu [21] classified normal and pathological skin lesions using four machine-learning algorithms. This method was able to obtain a 92.50% accuracy rate. Gradients and LBP were combined by Pereira et al. [22] for the lesion segmentation borderline characteristics. SVM–SMO (Sequential Minimal Optimization), SVM–ISDA (Iterative Single Data Algorithm), and FNN (Feedforward Neural Network) were the three classifiers that were used. In addition, they used two other datasets called Dermofit and MED-NODE. The MED-NODE performance metrics were 88%, 79%, and 65% for specificity, accuracy, and sensitivity.

Furthermore, Singh et al. [23] introduced a meta-learning approach to formulating low-data classification of medical images. This model was tested on several different medical datasets. They evaluated the proposed model using two benchmark datasets, "ISIC-2017 and PH 2". This model achieved an accuracy rate of 84.25% using ISIC-2018. Hasan et al. [24] presented an automatic skin lesion semantic segmentation. They reduced the network parameters using a depth-wise separable convolution. Astorino et al. [25] used a MIL algorithm to distinguish between melanomas and common nevi in some clinical data of dermoscopic color images. A computer-aided design for accurately diagnosing melanoma from dermoscopy images was described by Fu et al. [26]. They utilized the Kernel Fuzzy C-means method to select ROI. For the diagnosis of the images, an optimized classification algorithm based on a multi-layer perceptron was applied. They used Red Fox Optimization for feature selection. They achieved an accuracy rate of 90.5% on the ISIC-2020 dataset.

Deep learning

Esteva et al. [27] is the first research in which the Convolution Neural Network (CNN) was used in classifying skin lesions with a 72.1% classification rate. Pham et al. [28] proposed a system to identify and classify skin lesions using inception DL architecture with data augmentation. The classification rate for this model was 89%. Yu et al. [29] used a fully convolutional residual network (FCRN) to classify skin lesions with 85.5% accuracy. Wang et al. [30] recommended a bidirectional that modeled the complicated link between the relevant context information of the skin lesions and using a dermoscopic feature learning approach. A three-step method for detecting skin lesions that involve shrinking the image size has been reported by Amin et al. [31]. The feature selection based on the DNN model and iteration-controlled Newton–Raphson (IcNR) is combined to localize the skin lesion and recognition system. Then, the Otsu technique with the biorthogonal 2D wavelet transform was used to segment the lesions. Finally, features were extracted using a pre-trained deep model. Khan et al. [32] segmented skin images using Ant Bee Colony and used several DL models. They achieved a 93.4% accuracy rate with only two classes of ISIC 2017.

Several studies to classify skin lesions have been developed using various datasets. ResNet18 with different optimizers was used by Mahbod et al. [33]. Moreover, they extracted features of skin images from ISIC2017, and Various SVMs were used to classify these features with an 87.3% accuracy rate. Soudani and Barhoumi [34] used crowdsourcing for lesion segmentation in an autonomous skin lesion classification system to extract features. They applied transfer learning to ResNet50 and VGG16. They tested the proposed method using ISIC2017, where ResNet50 achieved 93.7%, 98.3%, and 83.3% for accuracy, specificity, and sensitivity, respectively. This research combined pre-trained deep networks such as “ResNet, AlexNet, GoogleNet, and VGG “ and transfer learning [35,36,37,38,39].

Yu et al. [40] proposed a system to recognize and classify dermoscopy images based on DCNN and the Fisher vector. This method was evaluated using ISIC2016 and achieved an 86.81% classification rate. Furthermore, based on the CNN and the ABCD rule, Almaraz-Damian et al. [41] extracted and fused features of skin lesions. Finally, Relevance Vector Machine (RVM), linear regression (LR), and SVM are used to classify these characteristics. The classification rate for this method is 89.71%. Majtner et al. [42] improved a melanoma detection and classification technique that combines DCNN feature extraction with linear discriminant analysis (LDA). They achieved an 86% classification rate. Albert [43] proposed a melanoma classification system in which lesions were segmented using Convergence of Intermediate Decaying OmniGradients (SCIDOG) and Synthesis. The ROI was classified Using the Predict-Evaluate-Correct K-fold (PECK) method, which combines DCNN, random forest (RF), and SVM. This approach provides a 91% categorization rate.

In addition, Al-Masni et al. [44] segmented the skin lesions and used several CNN architectures for classification. The classification rates for this method were 81.79%, 81.57%, and 89.28%, respectively, using ISIC2016, ISIC2017, and ISIC2018. Harangi et al. [45] recommended seven skin lesion classification systems using ISIC2018 and DCNN. This method can accurately classify 67.7% of the total lesion. Xie et al. [46] used mutual bootstrapping DCNN to segment and classify melanoma. They used the PH2 and ISIC2017 datasets to test their model, with 94% and 90.4% classification rates, respectively. Benyahia et al. [47] used 17 and 24 pre-trained DL to extract features and use ML to classify skin lesions. They evaluated their system using the ISIC2019 dataset, where they obtained a classification rate of 91.71% and 92.34% using cubic SVM and fine KNN, respectively.

Furthermore, Sarker et al. [48] introduced a lightweight GAN-based for skin lesion segmentation. They validated and tested their proposed model using ISIC-2017 and ISIC-2018, achieving an accuracy rate of 97.61%. Dai et al. [49] derived a multi-scale encoding and residual decoding network to segment skin lesion images. They evaluated their model using various datasets, ISIC 2016, ISIC 2017, ISIC 2018, and PH 2. Wibowo et al. [50] developed a lightweight encoder-decoder model using MobileNetV3-UNet for skin lesion segmentation, where this model was evaluated on ISIC-2017. Guo et al. [51] proposed a deep encoder and decoder network for medical image segmentation, where this model achieved an accuracy rate of 94.3% on ISIC-2017. Several studies have been performed in recent years for skin lesion segmentation and classification [51,52,53,54]. Hasan et al. [55] proposed a trained end-to-end CNN to recognize skin lesions. The two encoders extract features, and then these features are fused. The ISIC 2016 and ISIC 2017 are used in evaluating this model, where the achieved accuracy is 85.0% and 80.0%, respectively. Kassem et al. [56] removed the last three layers from GoogleNet to classify ISIC 2019, with an accuracy is 92.99%.

Cassidy et al. [57] introduced a strategy for duplicate removal. They applied the duplicated removal strategy with several DL architectures such as "DenseNet(121; 169; 201), EfficientNetB(0–4), InceptionResNetV2, InceptionV3, ResNet(152; 101; 50), ResNet(152; 101;50)V2, VGG (16; 19), and Xception". They evaluated the proposed model on ISIC-2020 and achieved 98.37% in classifying lesions as benign and malignant. The higher classification rate was 80.67% using VGG19. Sayed et al. [58] introduced a skin lesion classification system using the ISIC-2020 dataset. They used transfer learning to pre-trained learning, such as SqueezeNet. Also, they optimized the pre-trained model using bald eagle search optimization. Finally, a random over-sampling method and data augmentation were used to severe class imbalance. Using DL methods, Khatib et al. [59] produced a system that could identify skin lesions. They presented a decision-making system that uses various classifiers, including neural networks and feature-based methods.

Proposed method

The above approaches showed that localization information would get more discriminative features inside lesion areas. The discriminative features can be used to improve classification performance. The above classification methods used CNNs have a significant locating capacity with the class activation map. The class activation map was used to discriminate the discriminative ROI. Instead of part lesion localization using the pre-trained models, Skin-Net's proposed method can better localize the ROI using different filter sizes and a cross-channel correlation that ignores the spatial dimensions using a 1 × 1 convolution through the residual block.

The Skin-Net annotates classification level expressly to focus on intraclass distance minimization based on global characteristics in the same class. As shown in Fig. 1, images contain many artifacts such as air bubbles, hair, ruler, and high similarity between different lesions. All these challenges are addressed in the proposed model.

Fig. 1

Different lesion shapes and types

Skin-net architecture

If too few features are used, the classification will be inaccurate. So, the discriminative features are primarily for a successful classification process. To provide an accurate method for a challenging dataset of skin lesion images, we proposed a deep learning method using different criteria such as residual learning, cross-channel correlation, and imbalanced dataset serving, making the false-negative rate value less than state-of-the-art. A novel DL architecture called Skin-Net, including 54 layers, is suggested. The input, convolutional, pooling, batch normalization, dropout, fully connected, SoftMax, and activation rectified linear unit (ReLU) layers are some used to build the proposed residual learning method. The first layer's (i.e., "input layer") primary goal is to specify the dimensions of the input image, which correspond to the Height(H), Width(W), and the number of channels (D). In addition, there are three channels for RGB color images. Limited values for W and H, in contrast to the state-of-the-art DL models such as Res-net, Alex-net, Google-net, and VGG, “227 × 227” or “224 × 224” are used. In the proposed model, “W × H × D,” all images were resized to 300 × 300 × 3.

The “convolution layer” is the second layer, which takes its input from the previous layer, the input layer. This layer’s neurons link image subregions or even the layer output. Low-level features are used to learn convolution layers first, then additional features like objects, forms, and colors are extracted from going depth in layers. The features are localized after scanning the image by the regions learned from the convolutional layer. So, a downsampling approach is used to handle this issue. The downsampling approach must follow the convolutional layers, reducing the input image's features map by extracting the most relevant and crucial aspects. Because of this, we introduce a max-pooling layer. This layer generates a new group of features with the same features as the pooled features map.

During the training process, the layer parameters are changed, changing the input dimension of each layer. As a result, training the DCNN is difficult. Every iteration necessitates a slower learning rate and careful parameter initialization, which delays the training process. We used batch normalization layers to solve these problems, accelerate the training process, and reduce sensitivity. The batch normalization layer stabilizes learning by normalizing the input distributions' mean and variance, eliminating the harmful consequences of the internal shift covariance. A normalization layer exists before the ReLU modifies the DCNN [60]. As a result, the difficulty of coordinating updates between layers is alleviated.

Using global information is impractical while using the SGD method. Stochastic gradient descent (SGD) is computed in the condition of a small or random group of images. The SGD optimizer performs well using a low learning rate [61]. SGD maintains a single learning rate (alpha), which does not change during training. Instead of SGD during the training process, the ADAM (Adaptive Moment Estimation) optimizer was utilized to update network weights [62]. Each network weight (parameter) has its learning rate and is adjusted as learning progresses. The network parameters were randomly initialized and changed during the training process. The image features map was trained with ADAM using the Eqs. (2, 3):

$${m}_{t}= {\beta }_{1}{m}_{t-1}+(1-{\beta }_{1})\left[\frac{\delta L}{\delta {\omega }_{t}}\right]$$

(1)

$${v}_{t}= {\beta }_{2}{v}_{t-1}+(1-{\beta }_{2}){\left[\frac{\delta L}{\delta {\omega }_{t}}\right]}^{2}$$

(2)

where w refers to weights, t is time, $\delta \mathrm{L}$ is the derivated loss function, $\delta {\omega }_{t}$ refers to the derived weight at a specific time t, ${v}_{t}$ is the summation of the past gradients, and finally ${m}_{t}$ aggregates gradients at current time t.

To normalize the activation, subtract the mini-batch and divide by the standard deviation. This method speeds up learning and optimizes training. Let G denote the size N of the training mini-batch. The following equation is how the normalization of the input training vector $x=({x}^{1},\dots .,{x}^{n})$ computed:

$${nor x}_{i}^{(m)}= \frac{{x}_{i}^{(m)}-{\mu }_{G}^{(m)}}{\sqrt{{\sigma }_{G}^{{(m)}^{2}}+}\varepsilon }$$

(3)

where $m\in \left[1, Dimensional\, of\, the\, input\right]$, $i\in \left[1, n\right]$, where the pre-dimension of mean and variance ${\mu }_{G}^{\left(n\right)}$, and ${\sigma }_{G}^{{\left(n\right)}^{2}}$ respectively. The symbol $\varepsilon$ is a minor constant utilized to ensure numerical stability. The mean and variance may become 0 Without $\varepsilon$.

Raising and deepening layers is difficult and fraught with difficulties. The gradient disappearing problem can occur in deeper networks even with careful initialization. Experiments have shown that degradation is unavoidable and that increasing depth does not affect accuracy [63]. As a result, as network thickness increases, efficiency does not improve significantly and may be affected by the degradation problem. Training deeper systems requires more images, as these networks have many parameters to be called to generalize. Deep neural network layers are organized sequentially, and the output from one layer is fed into the next. So, going deeper is not a solution, as there are few images in the available datasets in medical imaging, especially in skin lesions. The proposed model uses a residual learning technique to overcome image shortages and degradation [64]. Residual learning addresses degradation by improving the flow of information and reformulating layers by skipping layer input connections. The proposed deep residual network is created by combining different residual blocks. The overall architecture of the proposed method is shown in Fig. 2. At the same time, the description for all convolutional layers is visualized in Fig. 3.

Fig. 2

The proposed deep residual network

Fig. 3

Convolutional layer details

As illustrated in Fig. 2, we create two types of residual blocks. The first skip connection is without any additional layer, such as the first, second, fourth, and sixth skip connections in Fig. 2. In contrast, the second skip connection consists of convolutional and batch normalization layers, such as the third and fifth layers. For each residual block, "x" is used to denote the input vector, the output vector is "y," and T(x) denotes the mapping of the stacked layers. Using the equation below, the residual function of these layers is now determined:

(4)

In Eq. (4), residual learning is employed to improve the layers learning rate, even with a limited dataset, instead of resembling the stacked layers ${T}_{i}(x)$. As shown below, Eq. (5) solves the degradation problem by establishing a shortcut link from the input to the stacked layers.

(5)

The stacked-layer output vector may have different dimensions than the shortcut connection vector “x.” So, the fusion of these two vectors is impossible (i.e., the dimensions must be the same for

and x).

A linear projection. ”${\mathrm{W}}_{\mathrm{s}}$” is used in this case to the vector "x" through the shortcut link. Finally, the output will be

(6)

The nonlinear activation function of the ReLU layer is used 65]. The ReLU function is a thresholding operation to convert elements less than 0 to zero. According to gradient vanishing, sparse representation and computational simplicity are utilized after the convolutional and batch normalization layers. We used a fully connected layer to transform the input volume into an N-dimensional output vector.

$$fully\, connected \,layer: {\mathbb{R}}_{M}\stackrel{yields}{\to }{\mathbb{R}}_{N}$$

(7)

where ${\mathbb{R}}_{\mathrm{M}}$ denotes the volume of the input, and ${\mathbb{R}}_{\mathrm{N}}$ refers to the number of classes.

The suggested model has been modified to operate with multiclass classification; instead of the sigmoid, the fully connected layer is employed after a SoftMax layer. The SoftMax output is converted to the target class name in the last layer, the "output layer," which employs the entropy function. In the sigmoid, the probabilities must be 1, while in SoftMax, it may be one where the target class has a higher probability value than other class values in SoftMax. The only disadvantage of SoftMax is that If the number of "lesion" classes grows, it becomes computationally expensive.

The proposed model can extract multilevel features. The proposed model used different filters with different sizes, such as 3 × 3 and 1 × 1, which extracted skin lesion characteristics on different levels. Using different filter sizes prevents the proposed model from overfitting. It improves the performance of lesion classification, in contrast with pre-trained models that perform convolution on the spatial and channel-wise domain. The proposed model performs a cross-channel correlation and ignores the spatial dimension [66, 67]. A sliding dot product is another name for cross-correlation. Cross-correlation measures the similarity between portions of one image. Moving a filter mask over the image and computing the sum of products at each position is the process of cross-correlation. Cross-correlation is a result of the filter's displacement [68]. Because the change may not affect all component peaks, the cross-channel correlation is less sensitive to background oscillations and will be reduced accordingly. The correlation scales noise proportionately to information, maintaining a consistent ideal signal-to-noise ratio across the entire frequency range. When each channel of an input feature map has to be handled separately, and the output feature maps need to be combined later, cross-correlation is utilized, as shown in Fig. 4.

Fig. 4

The proposed cross-channel correlation

The main challenge of the medical dataset is class imbalance. Any method trained and tested using an imbalanced dataset results in a biased decision. According to the state-of-the-art, oversampling, downsampling, etc., were used to overcome this challenge. Overfitting can occur when oversampling repeats examples from the minority class in the training dataset. In contrast, undersampling removes samples from the majority class, which can result in the loss of information vital to a model.

Alternatively, the proposed method performs a solution to serve the class imbalance. We developed a bootstrap for the dataset to balance out the classes. It operates by regularly sampling with replacement and weighting the samples based on the number of images in each class. First, we sort the dataset images alphabetically according to the name of the classes. The total number of images and the number in each class is computed. We divided the count of each class image by the total number of images in the dataset, which will be the images’ weight.

Moreover, images in the same class have the same weight. The summation of all labels' weight equals 1. Instead of using images labeled alphabetically, the dataset is converted to a vector, and the label of the images will be a numeric weight. So, the weight of the class containing the maximum number of images is the smallest, while the class containing fewer images is the maximum weight. After the training, the weight of the classification layer of the SKIN-net is multiplied by the image weight in the image vector, as shown in algorithm 1. This method overcomes the problem of overfitting and underfitting without duplicating the samples in small classes or losing samples from large classes, in addition to giving a valuable meaning to the image by assigning images to weight instead of Image to label. But any change in the dataset images will lead to weight changes that may lead to reproducing the weights again.

Experimental results and discussion

This section describes the proposed method's performance evaluation and the dataset used for training and testing.

Datasets

To evaluate the proposed method, we used a challenge from the ISIC archive “ISIC-2019” [69,70,71] and “2020 SIIM-ISIC Melanoma Classification challenge” [72]. ISIC-2019 is a well-known dataset that contains 25,331 images. ISIC-2019 consists of ISIC-2018 “HAM10000” and the BCN_20000. HAM10000 contains images of size 600 × 450, while BCN_20000 contains images of 1024 × 1024. Melanoma (MEL), Melanocytic Nevus (NV), Basal Cell Carcinoma (BCC), Actinic Keratosis (AK), Benign Keratosis (BKL), Dermatofibroma (DF), Vascular Lesion (VASC), and Squamous Cell Carcinoma (SCC) are the eight classifications in the ISIC 2019. The distribution of lesion types is as follows: MEL is 4,522; NV is 12,875; BCC is 3,323; AK is 867; BKL is 2,624; DF is 239; VASC is 253; and SCC is 628. Table 1 summarizes the distribution of lesion types for ISIC-2019.

Table 1 Distribution of lesion types of ISIC-2019

“2020 SIIM-ISIC Melanoma Classification Challenge” was hosted on Kaggle in the Summer of 2020 [72]. This dataset was created by the International Skin Imaging Collaboration (ISIC). A total of 33,126 dermoscopic training images of distinct benign and malignant skin lesions from over 2000 patients are included in the dataset. Each image is assigned to people with a unique patient ID. Histopathology was used to confirm all malignant diagnoses, whereas expert agreement, longitudinal follow-up, and histopathology were used to confirm benign diagnoses. The Hospital Clínic de Barcelona, the Medical University of Vienna, the Memorial Sloan Kettering Cancer Center, the Melanoma Institute Australia, and the University of Queensland University of Athens Medical School provided the images. This dataset includes 7 classes. These classes are named “lentigo NOS,” “lichenoid keratosis,” “melanoma,” “Nevus,” “seborrheic keratosis,” “solar lentigo,” and “unknown,” and the distribution of lesion types was 44, 37, 584, 5193, 135, 7, and 27126 respectively. Table 2 summarizes the distribution of lesion types for ISIC-2020.

Table 2 Distribution of lesion types of ISIC-2020

Training options and measures

On an IBM PC equipped with a processor Core i7, a DDRAM 16 GB, in addition to a GPU card NVIDIA MX150.MATLAB 2018b 64-bit was used to code the proposed model. SGD maintains a single learning rate (alpha), which does not change during training. Instead of SGD during the training process, the ADAM optimizer was utilized to update network weights. Each network weight (parameter) has its learning rate and is adjusted as learning progresses. The network parameters were randomly initialized and changed during the training process. All trials have the same weight decay, batch size, momentum, and maximum epochs; their values are reported in Table 3. To accelerate the computation, we proposed to drop the learning rate if the learning loss is not enhanced after ten epochs.

Table 3 training options values

The proposed method's performance was assessed using quantitative and qualitative metrics. Five quantitative measures include accuracy, sensitivity, specificity, precision, and F-score. If accuracy is used only to measure a model's goodness, a model that classifies all testing samples into a class containing the largest number of images will have excellent accuracy. Still, this model won't provide any valuable information for us. So, we used other performance measures. The true positive rate is called sensitivity, quantifying how effectively the positive class was anticipated. Specificity, complementing sensitivity, or the true negative rate quantifies how effectively the negative class was anticipated. The percentage of examples allocated to the positive class that belongs to the positive class is summarized by precision. The F-score, also known as the F-measure, is a single score that combines precision and sensitivity to balance both objectives.

As a qualitative metric, the confusion matrix and the receiver operating characteristic (ROC) are used to visualize and assess the dependability of the proposed method. The following equation is used to calculate these measurements [73]:

$$\mathrm{Accuracy}= \frac{{t}_{p}+{t}_{n}}{{t}_{p}+{f}_{p}+{f}_{n}+{t}_{n}}$$

(8)

$$\mathrm{Sensitivity}= \frac{{t}_{p}}{{t}_{p}+{f}_{n}}$$

(9)

$$\mathrm{Specificity }= \frac{{t}_{n}}{{f}_{p}+{t}_{n}}$$

(10)

$$\mathrm{Precision }= \frac{{t}_{p}}{{t}_{p}+{f}_{p}}$$

(11)

$$\mathrm{F}-\mathrm{score }= \frac{{t}_{p}}{{t}_{p}+\frac{1}{2} {(f}_{p} + {f}_{n})}$$

(12)

where ${t}_{p}$, and ${t}_{n}$ refer to true positive and true negative, while ${f}_{p}$, ${f}_{n}$ is false-positive and false-negative. A true positive means an accurate event value prediction, while a false positive means an inaccurate prediction. A true negative denotes an accurate no-event value prediction, whereas a false negative denotes an inaccurate prediction.

Experiments and discussion

The proposed method was evaluated on the “ISIC-2019” and “2020 SIIM-ISIC Melanoma Classification Challenge”. We divided the datasets into 70%, 15%, and 15% for training, validation, and testing. The dataset images were resized to 300 × 300 pixels to match the input layer size; in addition, no pre-processing step has been carried out on the images, such as hair removal, enhancement, or segmentation.

The main challenge in the ISIC-2019 dataset is that the test set includes an additional class named unknown, which wasn’t presented in the training dataset. The main challenge of the “2020 SIIM-ISIC Melanoma Classification Challenge” is class imbalance. If the proposed method is trained and tested using the dataset images, the proposed method will be biased to the class containing more images. We use the proposed solution to serve these two challenges classifying outliers images and class imbalance. We used external images to add distinct outlier images (unknown), and some healthy skin photos from [74] were used to create the outlier images. For class imbalance, we replaced the label of images with the weight of the image label. Finally, the proposed method multiplies the weight of the network with the weight of the image label. The proposed method is not biased to the class containing the maximum number of images.

We apply different augmentation methods such as random rotation angel from 0: 360, random vertical and horizontal flips, and random vertical and horizontal shifts. For ISIC-2019, We proposed augmenting only the classes containing several images less than 1000, such as AK, DF, SCC, unknown, and VASC. Each class has been augmented separately. The number of images in augmented classes is summarized in Tables 4 and 5.

Table 4 Distribution of lesion types of ISIC-2019 before and after Augmentation

Table 5 Distribution of lesion types of ISIC-2020 before and after Augmentation

Finally, the average performance measures of the proposed method during the testing process using ISIC-2019 were 94.65%, 96.78%%, 70.78%, 72.56%, and 71.33% for accuracy, specificity, sensitivity, precision, and F-Score, respectively. While the average of the same measures were 99.05%, 99.42%, 96.57%, 96.57%, and 96.57%, respectively, using ISIC-2020. The obtained results are summarized in Table 6. The confusion matrix obtained from the proposed method during testing is shown in Figs. 5 and  6. From specificity, sensitivity, precision, and f-score, we can observe that the proposed method works well with the imbalanced dataset.

Table 6 Obtained measures of the proposed model

Fig. 5

ISIC-2019 Confusion matrix

Fig. 6

ISIC-2020 Confusion matrix

Discussion

From the previous result, the proposed model achieved the best measures using ISIC-2020. The performance of the proposed model with the ISIC-2020 is better than the performance using ISIC-2019 for several reasons. First, ISIC-2020 contains more images than ISIC-2019. ISIC-2020 contains about 10 thousand more images than ISIC-2019. Second, ISIC-2020 contains a smaller number of classes. ISIC-2020 contains only seven classes, while ISIC-2019 contains nine classes. Finally, the unknown class images in ISIC-2019 were collected from different resources, as previously mentioned. In ISIC-2020, unknown class images are found in the official dataset. All of these reasons made the performance of the proposed model better using ISIC-2020 than ISIC-2019 for training and testing. The proposed method is compared with [56, 59] using the same dataset, ISIC-2019. Table 7 summarizes the obtained results, while the roc curve is shown in Fig. 7.

Table 7 Comparison of performance measures using ISIC-2019 with state-of-the-art

Fig. 7

ROC curves for ISIC-2019

Table 7 shows that the proposed method’s accuracy outperformed the state-of-the-art accuracy. The proposed model achieves a higher performance measure than the method [56, 59]. These methods augment all nine classes, but the proposed method augments only the classes containing less than 10000 images; as discussed, these classes are AK, DF, SCC, unknown, and VASC. In addition, the proposed work in [59] merged all classes into two classes: malignant and benign. They also don’t tackle the outlier images. From the ROC and Measures comparison, the proposed method proves its reliability.

Various approaches and tools were used to classify ISIC-2020 skin lesions. These approaches only classify lesions into two classes, “benign and melanoma,” instead of the total number of classes, “7”. These methods also start with a pre-processing step. Still, our proposed method doesn’t make any pre-processing step in images. The findings of these approaches and the suggested method are summarized in Table 8, and the roc curve is depicted in Fig. 8.

Table 8 Comparison of performance measures using ISIC-2020 with state-of-the-art

Fig. 8

ROC curves for ISIC-2020

From Table 8, all methods use only binary classification without using it to classify all seven classes, in addition to extensive pre-processing techniques for image enhancement, segmentation, and Augmentation. The results of the RDCNN-based method beat the current methods, as shown in Table 8. Finally, the proposed method outperforms previous methods of skin cancer classification. In addition, the proposed method didn’t bias to a specific class containing many images. Finally, the proposed method classifies multitype of skin lesions instead of binary classification. The proposed method didn’t bias the majority class because of using an imbalanced dataset during training and testing. In contrast, The proposed method couldn’t operate with devices with limited memory and microdevices. The proposed method tested only for a skin lesion.

Conclusion

Challenges of Skin lesions classification motivate the author to propose a novel deep neural architecture with 54 layers based on residual learning. The proposed model can accurately extract relevant information even with datasets including a few images. The proposed method can extract multilevel features by different filter sizes, such as 3 × 3 and 1 × 1. These filters lead to extracting skin lesion characteristics on different levels. Using different filter sizes prevents the proposed model from overfitting. It improves the performance of lesions. In addition, the proposed model performed a cross-channel correlation and ignored the spatial dimensions. Cross-correlation is utilized when each channel of an input feature map has to be handled separately. The cross-channel correlation is less sensitive to background oscillations and will be reduced accordingly. The imbalanced dataset has been tackled by converting the dataset to a vector of images and weights instead of images and labels. But any change in the dataset images will lead to weight changes that may lead to reproducing the weights again. The proposed model is tested and evaluated on challenging datasets ISIC-2019 and ISIC-2020. The proposed method obtained 94.65%, 96.78%, 70.78%, 72.56%, and 71.33% for accuracy, specificity, sensitivity, precision, and F-Score, respectively, using ISIC-2019. While the average of the same measures were 99.05%, 99.42%, 96.57%, 96.57%, and 96.57%, respectively, using ISIC-2020. The suggested RDCNN classification model beat previous approaches. There are some limitations, such as running time, and the proposed method isn't lightweight. So, the proposed method cannot run on microdevices. The new RDCNN might solve various classification issues and identify various malignancies. Further, the proposed method needs to be generalized for different classification tasks.