Author: John Byun, Steven D Chang, Mingli Chen, Cynthia Chuang, Xuejun Gu, Melanie Hayden Gephart, Yusuke Hori, Hao Jiang, Mahdieh Kazemimoghadam, Fred Lam, Gordon Li, Lianli Liu, Weiguo Lu, David Park, Erqi Pollom, Elham Rahimy, Deyaaldeen Abu Reesh, Scott Soltys, Gregory Szalkowski, Lei Wang, Qingying Wang, Zi Yang, Xianghua Ye, Kangning Zhang 👨🔬
Affiliation: Department of Radiation Oncology, Stanford University, Department of Neurosurgery, Stanford University, Medical Artificial Intelligence and Automation (MAIA) Lab, Department of Radiation Oncology, UT Southwestern Medical Center, UT Southwestern Medical Center, Department of Radiation Oncology, Stanford University School of Medicine 🌍
Purpose: Accurate prediction of pain relief is crucial in determining the clinical effectiveness of Stereotactic body radiotherapy (SBRT) regimen for spine metastases. We propose a deep-learning framework by integrating clinical factors and embedding learned through large language model (LLM)to predict the pain-relieving potential of SBRT for spine metastases.
Methods: We retrospectively collected 160 spine SBRT cases with spinal metastases from our institutional CyberKnife (CK) database. The dataset was partitioned into three groups: 104 cases for training, 26 for validation, and 30 for independent testing. Each case included two data modalities of clinical factors and imaging reports. The proposed framework comprises three key components: (1) data augmentation for clinical factors, (2) LLM-assisted analysis of the imaging reports, and (3) a cross-attention-based transformer classifier. The first component involves augmenting the dataset to generate numerical feature scores, encoding information such as the primary malignant tumor type, spinal metastases location, and bone metastasis-related pain conditions. Concurrently, an LLM-based analyzer processes the narrative and impression in imaging reports, and the latent representations obtained from the LLM are directly embedded into the classifier to avoid potential information distortion caused by tokenization. Finally, both modalities of data are integrated into a cross-attention-based transformer, followed by a fully connected layer to predict the outcome of pain relief conditions.
Results: The proposed method achieved an accuracy of 85.15%, a sensitivity of 93.02%, a specificity of 79.31%, and an area under the curve (AUC) of 0.85. We also conducted an ablation study using the single modality of clinical factors. The preliminary results demonstrated improved performance with multi-modal data with the accuracy of 85.15%, exceeding the single modality 80.95%.
Conclusion: Our proposed framework enables multi-modal, LLM-powered prediction of pain relief outcomes for spine metastases SBRT. Preliminary results demonstrate promising performance, highlighting its potential to enhance and streamline the spine SBRT workflow.