Author: Mark Anastasio, Zong Fan, Hua Li, Changjie Lu, Lulu Sun, Xiaowei Wang, Zhimin Wang, Michael Wu 👨🔬
Affiliation: University of Illinois at Urbana-Champaign, University of Illinois at Chicago, Washington University School of Medicine, University of Illinois Urbana-Champaign, Washington University in St. Louis, University Laboratory High School 🌍
Purpose: Histological whole slide images (WSIs) are vital in clinical diagnosis. Although deep learning (DL) methods have achieved great success in this task, they often lack interpretability. Traditional machine learning methods can utilize hand-crafted features to provide interpretable analysis. This study aims to improve the interpretability and performance of histological image classification by integrating hand-crafted features with DL models through a Weak-to-Strong Generalization (WSG) approach. The proposed WSG workflow seeks to bridge this gap by combining explicit, interpretable hand-crafted features with robust DL models.
Methods: The WSG workflow has three stages. In Stage 1, multiple hand-crafted features like gradients, density, and shape are extracted from input WSI image patches. Decision tree models are trained to serve as weak supervisors. In Stage 2, a DL model is pre-trained in a self-supervised contrastive learning paradigm to create a strong model. In Stage 3, the strong model is fine-tuned using predictions from the weak model through an adaptive WSG loss function. This loss function incorporates cross-entropy and an adaptive weighting mechanism to balance the influence of the weak and strong models. This WSG image classification workflow can investigate the correlations between hand-crafted and DL features, providing deep insights into feature interpretability.
Results: The WSG approach was validated using the CAMELYON16 dataset, demonstrating improved classification accuracy for tumor versus normal tissue identification. The ResNet-18 model, fine-tuned using WSG, achieved an accuracy gain of 1.08%, while VGG-11 and MobileNet-v2 saw improvements of 1.19% and 1.91%, respectively. Feature importance analysis revealed hand-crafted features (e.g Nucleus Haralick SumAverage) significantly contribute to classification performance, highlighting the synergy between hand-crafted and DL features.
Conclusion: The WSG workflow enhances the interpretability and performance of histological image classification by effectively combining hand-crafted features with DL models. Our approach provides a new direction for research on interpretable AI models in medical imaging.