AI Powered Clinical Decision Support System for Radiology

¹ Department of Artificial Intelligence and Data Science, KPR Institute of Engineering and Technology, Coimbatore
² Department of Electronics and Communication Engineering, KPR Institute of Engineering and Technology, Coimbatore

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Published online: 5 March 2026

Abstract

Recent breakthroughs in AI technology have resulted in sophisticated tools designed to aid doctors by aiding them in making diagnoses, documenting cases, and formulating decisions. The initiative incorporates an AI-driven healthcare advisory tool for Radiology using models like OpenAI's Whisper and Google's Gemini APIs alongside NLP methods for automatically creating organized diagnostic summaries based on verbal inputs. Using an integrated framework comprising React for frontend development, Fast API as its backend engine, and Firebase for cloud-based data management, this system transmutes unstructured medical recordings of Radiology into structured diagnostic reports effortlessly. Speech recognition software by Whisper achieves precise text conversion through an adapted method utilizing about 500 recordings collected in medical speech transcriptions and intents datasets obtained from Kaggle, augmented with synthesized voice material produced by Google's TTS technology. Subsequently, the audio transcript undergoes processing through the Gemini API, converting it into organized medical records comprising elements like Initial Symptoms, Observations, and Diagnosis. The front-end provides users like physicians and radiologists with easy-to-navigate tools allowing them to input medical information quickly, watch live transcripts in progress, and get instant feedback on their work via artificial intelligence-assisted summaries. The FastAPI component oversees interactions among the front end, the Whisper AI engine, and the Gemini service, guaranteeing smooth audio processing and output creation. Firebase manages data retention and updates, ensuring both safe cloud based file management and instantaneous accessibility of healthcare records in real time. This comprehensive model markedly decreases laborious tasks requiring human effort, improves precision in diagnosis.