Author: Kristen A. Duke, Samer Jabor, Neil A. Kirby, Parker New, Niko Papanikolaou, Arkajyoti Roy, Yuqing Xia 👨🔬
Affiliation: St. Mary's University, The University of Texas San Antonio, UT Health San Antonio 🌍
Purpose:
The Segment Anything Model (SAM) is a foundational box-prompt-based model for natural image segmentation. However, its applicability to zero-shot 3D medical image segmentation, particularly in Radiation Oncology, remains underexplored and inadequately validated. To address this gap, we explored SAM based models in segmenting multimodal medical images on four distinct datasets including different anatomical structures, modalities, and targets. We have further fine-tuned SAM and developed SmartSAM, a novel method for automated segmentation of organs-at-risk (OARs) and brain gross tumor volumes (GTVs) in radiotherapy. SmartSAM employs a zero-shot approach and features a user-friendly web application for multimodal medical image segmentation.
Methods:
This study analyzed CT and MRI scans from 360 cancer patients across brain, lung, abdomen, and pelvis treatment sites using public datasets. Seventy percent of the cases were reserved for training, while the remaining were used for testing. The segmentation focused on 30 OARs and brain GTVs. Segmentation accuracy was evaluated using the Dice Similarity Coefficient (DSC) and Intersection over Union (IoU), with comparisons against manual delineations and other SAM-based models. A preliminary web application, designed using Python, JavaScript, and HTML, was developed to enable zero-shot usage by physicians and medical physicists.
Results:
SmartSAM achieved superior segmentation performance across diverse anatomical structures. Average DSC scores for abdomen CT, lung CT, pelvic CT, and brain tumor MRI images were 0.91±0.01, 0.89±0.03, 0.92±0.02, and 0.87±0.03, respectively, outperforming other state-of-the-art methods such as U-Net and SAM by 3–5% for selected OARs. SmartSAM also demonstrated improved efficiency as compared to the time required for manual contouring.
Conclusion:
SmartSAM demonstrates a robust, zero-shot solution for standardized and efficient auto-segmentation in research and clinical settings. By integrating high accuracy, time efficiency, and user accessibility, it enhances radiotherapy workflows and enables precise segmentation of multimodal medical images, facilitating improved treatment planning and broader clinical applications.