Diagnostic Performance and Concordance of a BI-RADS-Output AI System in Mammography: A Retrospective Study of Radiologists-AI Collaboration.

The identification of breast cancer using mammography and ultrasonography is influenced by inter-observer variability. Artificial intelligence (AI) models for breast classification have been investigated, but limited reports exist on Breast Imaging Reporting and Data System (BI-RADS)-output AI systems. This study attempted to assess a BI-RADS-output AI system by comparing breast cancer detection rates among radiologist-only, AI-only, and combined interpretations, while also focusing on the AI's ability to identify cancers missed by radiologists and analyzing diagnostic concordance concerning imaging features and BI-RADS categories. A retrospective analysis was performed on 331 lesions from 321 consecutive breast cancer patients who underwent preoperative digital mammography and ultrasonography. McNemar's test was used to find the detection rates, and chi-square/Fisher's test, Receiver Operating Characteristic Curve (ROC) analysis, and Kappa statistics were used to verify the diagnostic performance. The combined radiologist-AI interpretation achieved the highest cancer detection rate of 90.6%, which was higher than the radiologist's rate of 87.6% and the AI's rate of 84.0% (P < 0.05). The advantage was especially evident for BI-RADS 4B lesions, where AI alone decreased to 54.4%, while the combined method increased to 81.8%. AI found tumors that radiologists missed, with an area under the receiver operating characteristic curve (AUC) of 0.840. There was a strong agreement for microcalcifications (κ = 0.827), but only a weak to moderate agreement for mass morphology and final BI-RADS classification (κ = 0.142-0.414). Although AI-radiologist agreement varied across different imaging features, their complementary strengths enabled improved cancer detection performance, supporting the role of BI-RADS-output AI systems as clinical decision-support tools rather than replacements for radiologists.