Author: Chuangxin Chu, Haotian Huang, Tianhao Li, Jingyu Lu, Zhenyu Yang, Fang-Fang Yin, Tianyu Zeng, Chulong Zhang, Yujia Zheng 👨🔬
Affiliation: The Hong Kong Polytechnic University, Nanyang Technological University, Australian National University, Medical Physics Graduate Program, Duke Kunshan University, North China University of Technology, Duke Kunshan University 🌍
Purpose: Deep learning segmentation models, such as U-Net, rely on high-quality image-segmentation pairs for accurate predictions. However, the recent increasing use of generative networks for creating synthetic images poses a significant challenge. Without rigorous quality control, synthetic data introduced into training processes may substantially degrade model performance. This study systematically evaluates the impact of synthetic MRI data, viewed as a form of “data poisoning,” on the robustness and segmentation accuracy of U-Net models for brain tumor segmentation.
Methods: A U-Net model was trained on T1-contrast-enhanced MRI scans from 150 glioma patients for enhancing tumor (ET) segmentation. Synthetic data was generated using an innovative GAN-based model with a shared encoding-decoding framework and shortest-path regularization. This GAN was trained on paired CT-MRI datasets to convert CT scans into synthetic MRI images. To assess the negative impact of synthetic data on segmentation performance, different proportions (16.67%-83.33%) of synthetic T1-Ce data were mixed into the training set, creating “poisoned” datasets. Two types of models were compared: (1) a baseline U-Net trained on real MRI data and (2) poisoned U-Nets trained with mixed real and synthetic data. All models were tested on a real MRI validation set, with segmentation performance assessed using Dice coefficient, Jaccard index, accuracy, and sensitivity.
Results: Segmentation performance deteriorated significantly as the proportion of synthetic data increased. Dice coefficients decreased to 0.8937±0.0722, 0.8572±0.1580, 0.8146±0.2457, and 0.7474±0.2650 for synthetic data proportions of 33.33%, 50.00%, 66.67%, and 83.33%, respectively. Accuracy and sensitivity followed similar downward trends, demonstrating a notable decline in segmentation precision and robustness.
Conclusion: The introduction of synthetic MRI data into training sets significantly compromises the robustness and segmentation accuracy of U-Net models as the scale of synthetic data increases. The study highlights a crucial need for best practices in handling synthetic data to mitigate risks in future segmentation tasks.