Quantitative Evaluation of the Diagnostic Accuracy of ChatGPT in the Adult Chest Computed Tomography Images: A Phantom Study

Abstract

This study aimed to assess the quantitative diagnostic accuracy of ChatGPT in interpreting adult chest computed tomography (CT) phantom images using contrast-to-noise ratio (CNR) as the reference standard. A quantitative descriptive–comparative research design was employed, utilizing thirty (30) CT phantom images acquired under adult chest CT protocols with varying milliampere-seconds (mAs). Objective image quality was evaluated through computed CNR values, while the same images were assessed by ChatGPT using a standardized evaluation approach. The results of both methods were statistically compared using a paired t-test. Findings revealed that quantitative CNR values exhibited wide variability, including both positive and negative results, reflecting sensitivity to changes in image quality. In contrast, ChatGPT-generated values were consistently positive and showed minimal variation, indicating stable but generalized outputs. Comparative analysis demonstrated that ChatGPT consistently produced higher evaluation scores than the quantitative method. The paired t-test confirmed a statistically significant difference between the two methods (p = 0.002). These results suggest that ChatGPT lacks sensitivity to variations in CT image quality and tends to overestimate results. While it may serve as a supplementary tool, quantitative evaluation remains essential for accurate assessment of image quality.