Author: Michael Dohopolski, Xuejun Gu, Hao Jiang, Steve B. Jiang, Christopher Kabat, Jingying Lin, Weiguo Lu, Michael Tang 👨🔬
Affiliation: Medical Artificial Intelligence and Automation (MAIA) Lab & Department of Radiation Oncology, UT Southwestern Medical Center, Neuralrad LLC, Medical Artificial Intelligence and Automation (MAIA) Lab, Department of Radiation Oncology, UT Southwestern Medical Center, UT Southwestern Medical Center, Department of Radiation Oncology, Stanford University School of Medicine 🌍
Purpose: To streamline access to clinical data stored in Oncology Information Systems such as MOSAIQ or ARIA, we developed an AI-powered chatbot capable of querying, summarizing, and interactively answering questions about patient information to enhance clinical decision-making and improve workflow efficiency.
Methods: The chatbot integrates a local large language model (LLM) and an Qdrant vector database. It uses a MySQL database to store treatment data. Using a MOSAIQ data dictionary, SQL queries were designed to extract patient information. Patient IDs were determined by querying the Ident table using the Medical Record Number (MRN) provided through a Svelte.js frontend.
Additionally, MOSAIQ includes document files, such as PDFs, converted into ASCII format and stored alongside structured data in MongoDB. Both structured and document-derived content were embedded into a Qdrant vector database for semantic search. The backend, built with Python Flask, handled database querying, data curation, and embedding using a Sentence Transformer model. A locally deployed Microsoft Phi-4 model, a 14B parameter LLM running on an Nvidia GeForce 4090 GPU, generated patient summaries and answered queries. Selecting a patient in the frontend produced a concise summary, and users could ask questions about diagnoses, treatment history, clinical trials, or document contents, with the LLM providing context-aware responses.
Results: We tested the chatbot on 57 patients with disease sites including lung, breast, and prostate cancer. The chatbot achieved an accuracy of ~95% for retrieving patient-specific information and summarizing document content. The retrieval time for patient data ranged from 0.8 seconds to 2.5 seconds, depending on query complexity and the size of the dataset accessed.
Conclusion: The chatbot’s integration of database querying, vector embedding, and LLM inference provides a powerful tool for automating data retrieval. Future work will focus on scalability and additional functionalities to support oncology practices.