A clinical AI teammate finds toxicities—but it earns its place by helping people do the work

October 29, 2026 · Prospective silent deployment + randomized staff crossover; retrospective validation · Primary source

Cancer immunotherapy can work remarkably well, but immune-related adverse events may emerge in clinical notes before they appear in a clean research database. An October 29 Nature Cancer study tested a narrower, more useful ambition for an AI agent: identify, characterize, and help people verify possible treatment toxicities. It is an example of AI designed around a real clinical research task rather than a promise to replace a clinician.

On retrospective records, the system achieved a macro-F1 of 0.92 for detecting current events, but only 0.66 for classifying multiple severity grades. During three months of silent prospective deployment, binary F1 dropped to 0.72–0.79. In a randomized crossover evaluation of trial staff, assistance cut annotation time by 2.71 minutes per note, increased the odds of an exact complete match (OR 1.45, 95% CI 1.01–2.08), and improved inter-rater agreement. The authors estimated about $0.02 per note for the best configuration, a model-use cost rather than the full implementation cost.

The distinction between retrospectively optimized accuracy and performance on incoming notes matters. The trial measured annotation quality and speed—not whether an oncologist made fewer harmful treatment decisions or whether patients lived longer. Yet these are measurable, realistic gains.

Clinical reading: Worth a supervised research-workflow pilot; not autonomous toxicity triage or independent patient management.