Deep Neural Networks with Transfer Learning for Medical Image Classification

Vidya Prathishthans Arts, Science & Commerce college, Baramati 413133 Maharasthtra

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 20 November 2025

Abstract

Medical image classification is a crucial component in the aid of clinical diagnosis and disease screening. Deep learning models usually suffer from a lack of data as well as low interpretability in the medical imaging context. In an effort to mitigate these issues, this paper suggests a Hybrid CNN-Transformer Transfer Learning Model (ResViT-MedNet) that combines the ability of Convolutional Neural Networks (CNNs) to extract local features with Vision Transformers (ViTs)' global contextual knowledge. The model exploits transfer learning with a ResNet-50 backbone pre-trained on ImageNet, combined with Transformer encoders and an attention-fusion scheme based on attention to harmonize local and global representations. Experiments were carried out on the ISIC 2020 Melanoma Classification Dataset of 33,126 dermoscopic images. The model proposed in this study attained an accuracy of 98.47%, precision of 0.982, recall of 0.984, F1-score of 0.983, and AUC of 0.994 and outperformed baseline architectures like VGG16, ResNet50, ViT-Base, and EffNet-ViT. The improved classification performance and better interpretability show that ResViT-MedNet indeed resolves overfitting issues and generalizes well on small medical datasets. These findings showcase its promise as a trustworthy clinical decision-support system for autonomous skin lesion classification.