Author: Wei Wei, Yading Yuan 👨🔬
Affiliation: Columbia University Irving Medical Center, Department of Radiation Oncology 🌍
Purpose: To investigate an uncertainty modeling method to improve the performance of cancer classification with the ability to produce uncertainty score.
Methods: Deep learning has achieved state-of-the-art performance in many medical image analysis applications. However, they usually lack the ability to quantify the uncertainty of their predictions, which hinders their reliability in clinical use. In this study, we propose a novel uncertainty learning framework that enables producing both the class label and the uncertainty score associated with the prediction. Our novelty lies in leveraging useful discriminative information from hard samples to facilitate the learning of uncertainty in relative dynamics, instead of predicting the uncertainty of each sample independently. We demonstrate the effectiveness of our method using two open-source skin cancer datasets ISIC 2018 and 2019. We showcase the superiority of our method compared to state-of-the-art methods, in terms of both generalization on unseen images and reliability of uncertainty quantification.
Results: Comparing to the deterministic methods without estimating uncertainty, we achieve the best performance under most of the metrics (Accuracy, Sensitivity, Specificity, F1-score, and AUC-ROC) for both datasets. Notably, we improve the metrics Sensitivity and F1-score significantly over comparison methods (+~5%). Meanwhile, comparing to other uncertainty modeling methods, our method achieves better scores under all uncertainty quantification metrics (negative log-likelihood (NLL), expected calibration error (ECE), and Brier-score).
Conclusion: Our preliminary results on the task of skin cancer classification demonstrate that the proposed uncertainty modeling method which leverage the discriminative hard samples can improve both generalization and reliability.