The AI can read uncertainty in a radiology report. Can it tell us what that means?

Clinical reports contain more than findings. They communicate confidence, ambiguity and the limits of interpretation. A November 3 feasibility study tested whether two large language models could place the wording of chest radiograph reports on a four-point uncertainty scale. This is exactly the kind of narrow document task that can look remarkably good in aggregate while failing at the edge cases that matter.

From a public corpus of 3,927 reports, three radiologists independently graded 100 and adjudicated disagreements. Claude Opus 4.5 and Gemini 2.5 Pro showed similar quadratic weighted kappa agreement with the reference standard, 0.807 and 0.814. Yet in the eight most ambiguous reference reports, Claude exactly identified five while Gemini identified only one. These are very small subgroup denominators, and some disagreements involved an adjacent grade.

The model classified linguistic uncertainty, not the patient’s risk of disease. Radiologists themselves showed only moderate pairwise agreement. The study included no clinical intervention, no reduction in diagnostic errors and no evidence of safer triage. Treating its high aggregate score as clinical readiness would be a category error.