Skin cancer classification from dermoscopic images based on Convolutional Neural Network and Vision Transformer

Skin cancer poses a vast worldwide health issue because of its fast progression and high mortality rate. Early diagnosis and precise prognosis are important for the treatment of these patients. In this paper, we propose an enhanced hybrid deep learning model composed of a combination of Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) for the dermoscopic image classification task. The designed architecture adopts ResNet50, EfficientNet, and ViT as strong feature extractors with an innovative attention module, MambaATT, for precise modelling and utilisation of global context information. Methods for statistical growth (rotation, flips, and scaling) have been tested to improve the model's generalisation and its variety, even if the dataset is short. When evaluating the entire dataset, we found that our proposed version CNN_ViT_MambaATT reaches an advanced-class accuracy of 92, outperforming the traditional methods for individual networks. The findings highlight the power of combining strong CNN and VIT models for both mechanisms of interest and information growth strategies, offering a robust and accurate approach to early skin cancer analysis.

Text 126583.pdf - Published Version Restricted to Repository staff only Download (1MB) Official URL or Download Paper: http://www.pertanika.upm.edu.my/pjst/browse/regula...

Actions (login required)

View Item