Author: Xiangli Cui, Chunyan Fu, Man Hu, Wanli Huo, Jingyu Liu, Jianguang Zhang, Yingying Zhang, Shanyang Zhao 👨🔬
Affiliation: Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, the Zhejiang-New Zealand Joint Vision-Based Intelligent Metrology Laboratory, College of Information Engineering, China Jiliang University, Departments of Radiation Oncology, Zibo Wanjie Cancer Hospital, Key Laboratory of Electromagnetic Wave Information Technology and Metrology of Zhejiang Province, College of Information Engineering, China Jiliang University, Department of Oncology, Xiangya Hospital, Central South University, College of Information Engineering, China Jiliang University, Department of Radiation Oncology, Shandong Cancer Hospital and Institute, Shandong First Medical University, Shandong Academy of Medical Sciences 🌍
Purpose: To quantify the impact of fine-tuning strategies for pre-trained AI image generation models on glioma MRI image quality and observer performance, and to determine the optimal fine-tuning configurations and the minimum dataset size needed for generating high-quality images.
Methods: A glioma MRI generation model was developed to investigate the feasibility of fine-tuning AI models for rare diseases. Fine-tuning was applied to the CLIP text encoder and the U-Net noise prediction network. The CLIP encoder, based on contrastive learning, provided cross-modal semantic information, while the U-Net used this conditional information to predict noise as a denoising module. The U-Net parameters were adapted to the radiological domain using MSE loss and the Adam optimizer. Textual inversion was employed to extract glioma-specific cross-modal features. The indistinguishability between generated and real images was quantified using human observers' signal-to-noise ratio (SNR) via a two-alternative forced-choice (2AFC) test. Image quality was further evaluated using t-SNE visualization. Multiple imaging modalities were studied, each highlighting different anatomical features such as tumors, edema, and cerebrospinal fluid. Performance was assessed using multiple metrics across these modalities.
Results: Our method performed well across modalities with as few as 50 images per dataset. Performance remained stable when the dataset size was reduced to one-fifth of the original size, although T2-weighted imaging, which has inherently weaker signals, showed a relative decline. For simpler tasks with high-quality data and prior knowledge (pretraining on normal MRIs), performance was preserved even when the dataset size was reduced to one-tenth.
Conclusion: This study demonstrates the potential of fine-tuning pre-trained AI models to generate rare disease MRIs with significantly smaller datasets than traditional methods. Observer performance was unaffected by reduced dataset size, and prior pretraining further reduced the data requirements. This approach can be extended to more complex imaging tasks and other medical modalities in future research.