Author: Kostas Danniidis, Agelos Kratimenos, Yufu Wang, Timothy C. Zhu, Yifeng Zhu 👨🔬
Affiliation: University of Pennsylvania 🌍
Purpose: This study aims to develop software and algorithms utilizing artificial intelligence (AI) to seamlessly create 3D patient postures during Total Skin Electron Therapy (TSET). The resulting mesh will provide a foundation for the cumulated surface dose evaluation based on Monte-Carlo simulation for individual patients.
Methods: To model the human mesh, we use the SMPL model, a skinned vertex-based framework that accurately represents diverse human body shapes in realistic poses. It models a body using 10 shape parameters for anatomy, 23 joints with 3 rotational degrees of freedom each, and 3 degrees of freedom for global (root) orientation. Given these parameters, we can compute the 3D coordinates of 6890 vertices of a human mesh, with consistent indexing across postures. To initialize optimization, we use BEDLAM to predict an SMPL model from a grayscale Cherenkov image taken during the treatment. Given 6 images of 6 postures of a patient, we use SAMv2 to obtain a 2D contour of the silhouette, and then use VITPose, a transformer-based model, to predict 2D body keypoints for each pose. Further, we optimize joint angles, root orientation, and shape coefficients by minimizing deviations between the 3D model’s projections and the 2D images and finally rescale the models to match 3D scan data for accurate dose calculations.
Results: The silhouettes and joint positions in all 6 views encode all information we need to solve the inverse problem of obtaining the shape parameters and the 23 joint plus the root orientation that fully determine the anatomy and posture of the human. We are able to generate the 6 models for each patient based on Cherenkov images and make cumulative MC dose evaluations.
Conclusion: Preliminary results suggest that AI-assistance can be successfully utilized to recreate adequate patient postures to provide valuable dose distribution analysis in TSET.