Artificial intelligence could help support less-experienced clinicians in identifying skin cancer but it still performs more poorly compared with expert dermatologists, research suggests.
The findings, in JAMA Dermatology, suggest that the oft-reported superiority of AI in diagnosing skin cancer may need closer inspection in situations that are more similar to daily clinical practice.
First-generation convolutional neural network (CNN) systems did not maintain their reported advantages when confronted with a broad spectrum of cases, including rare and atypical presentations.
Foundation models were more promising, reproducing a substantial portion of clinical expertise and approaching the diagnostic accuracy of well-trained clinicians while surpassing that of novices.
Nonetheless, these models still fell short of the best experts who had at least a decade of experience.
“This shows that human expertise at the highest level remains indispensable and that experience continues to be the most powerful tool for performance,” reported Luc Thomas, PhD, from Hôpital Lyon Sud in France, and co-workers.
They suggested: “AI tools may be most valuable as decision-support systems for less experienced clinicians, effectively functioning as a virtual mentor.”
The prevailing narrative suggests that AI has matched or surpassed human expertise in medical diagnosis, particularly in the imaging-based specialties.
Yet a substantial gap remains between promising results under controlled experimental conditions and meaningful clinical implementation, which requires integrating factors such as patient demographics, medical history, physical findings, and contextual information.
To get a better comparison in realistic clinical settings, Thomas and team compared the diagnostic performance of 652 physicians with varying dermatological expertise with three AI algorithms: a first-generation CNN model and the PanDerm uni- and multimodal foundation models.
The dataset comprised dermatological images—including clinical and dermoscopic images with associated metadata—from 1117 cases that represented everyday clinical scenarios.
Results showed that expert dermatologists with at least a 10 years’ experience achieved the highest multiclass accuracy, at a mean of 74.2%, outperforming all AI models on this primary endpoint.
The lowest accuracy was for CNN, at 56.7%, while unexpectedly the modern unimodal foundation model outperformed the multimodal version, at a corresponding 72.2% versus 66.3%.
All human readers outperformed the CNN, with the former collectively having an accuracy of 65.9%. However, the unimodal model was better than that of readers with less than a year of experience and those with less than three years of experience, who had accuracies of 59.1% and 68.2%, respectively.
Among the malignant lesions missed by both foundation models, there appeared to be a preponderance of acral localizations.
“The future likely lies in collaboration between humans and machines to optimize diagnostic performance,” the researchers concluded.
“For novice practitioners, AI could serve as a safety net and educational tool. For experts, it could provide an efficient triage modality and a systematic second reading, particularly useful for reducing errors caused by fatigue or inattention.”
The post Experienced Physicians Still Beat AI at Skin Cancer Diagnosis appeared first on Inside Precision Medicine.