Author: Matthew S Brown, Joshua Genender, John M. Hoffman, Gabriel Melendez-Corres, Muhammad W. Wahi-Anwar 👨🔬
Affiliation: David Geffen School of Medicine at UCLA, UCLA Department of Radiology 🌍
Purpose: Renal lesions are evaluated using scoring systems based on visual assessments and manual measurements. The purpose of this work is to develop a multi-agent system for automated anatomic landmark and nephrometry feature extraction on CT.
Methods: Data from a public dataset, containing 489 CT images, corresponding masks for the kidneys and lesions, and clinical data, was split into 70/30 training and test sets. A Cognitive AI framework, SimpleMind, was used to build a multi-agent system. This approach allows the aggregation of segmentation agents and a layer that can reason with their outputs. This reasoning is achieved with agents for feature extraction, spatial inferencing, and fuzzy logic. Agents are configured using a knowledge graph (KG), where each node represents an agent and its parameters, and links represent results being passed between agents. For each image-mask pair, kidney and lesion candidates are post-processed and checked for anatomic consistency. From the kidney masks, relevant anatomic landmarks for nephrometry, such as the urinary collecting system, are derived by spatial inferencing agents. Feature extraction agents take lesion and landmark masks as inputs and compute features used in nephrometry scoring, including lesion diameter, endophycity, distances, overlapping surfaces, etc. To evaluate the utility of the extracted features to predict lesion malignancy and necrosis, Logistic Regression (LR) classifiers were trained and tested against clinical data.
Results: The SimpleMind KG successfully extracted landmarks and features across all scans. For malignancy prediction, LR yielded a mean AUC of 0.53±0.10 and 0.61±0.07 on the validation and test sets respectively. For necrosis prediction, LR yielded a mean AUC of 0.89±0.11 and 0.80±0.01 respectively.
Conclusion: We developed a multi-agent system that can aggregate segmentation agents and reason with their outputs to automate visual nephrometry scoring. It makes these assessments more feasible in clinical practice and provides explainable decision support using interpretable features.