Author: Ramesh Boggula, Jay W. Burmeister, Michael Joiner 👨🔬
Affiliation: Wayne State University, Karmanos Cancer Center, Gershenson ROC, Wayne State University School of Medicine 🌍
Purpose: Recent advances in large language models such as ChatGPT offer new possibilities for supplementing traditional teaching methods. In this study, we developed a custom GPT-powered tool freely accessible at https://chatgpt.com/g/g-h2Brl0pXv-medical-physics - that can respond to a wide range of medical physics-related queries.
Methods: We created a custom Medical Physics GPT using the OpenAI GPT Store, a platform that allows users to develop specialized conversational AI tools tailored to specific domains. The model was fine-tuned specifically on medical physics related material. Fifteen participants (5 undergraduate radiation therapy students, 8 graduate medical physics students, and 2 radiation oncology residents) were invited to test this new Medical Physics GPT. After using the chatbot, each participant completed a short survey that rated four categories on a scale from 0 (very poor) to 5 (excellent). The categories were: 1) the accuracy of the chatbot’s responses, 2) the clarity of explanations for complex topics, 3) the relevance of the answers to their academic and clinical needs, and 4) overall satisfaction with the tool. We compiled and analyzed these survey responses to assess the tool’s effectiveness.
Results: Overall, participants gave predominantly high ratings across all four categories, with most ratings at 4 or 5. Specifically, the average (± SD) scores were: accuracy: 4.9 ± 0.4, clarity: 4.6 ± 0.6, relevance: 4.8 ± 0.4 and overall satisfaction: 4.8 ± 0.4. Some participants appreciated that the chatbot's responses were detailed, while others felt that understanding the extensive information and verifying each specific response could be time-consuming. This feedback suggests that balancing depth and conciseness may be critical for optimizing the learning experience.
Conclusion: These preliminary findings suggest that a GPT-based tool can effectively complement traditional medical physics education. Future research will include larger participant groups and improvements to broaden the tool’s capabilities.