The evidence review can fail before the statistics begin

Source: JAMA Network Open, online October 29, 2026; systematic audit of Cochrane diagnostic meta-analyses, article and PubMed, DOI 10.1001/jamanetworkopen.2026.40850. Evidence: methodology audit, not a new clinical intervention.

Our whole research system rests on a simple promise: when we report the effect of a treatment or accuracy of a test, the underlying numbers still mean what the original investigators measured. A new systematic audit reveals how easily that promise can break. Researchers reviewed 46 systematic reviews, 879 diagnostic comparisons, and the primary data from 6,173 studies. Data extraction could not be reproduced for 338 studies (5.5%); errors affected 137 quantitative diagnostic meta-analyses and 34 of 46 reviews (73.9%).

Not every error changed the bottom line. Among the 137 reproduced meta-analyses, substantial changes occurred in about 0.7% for sensitivity, 2.9% for AUC and 3.6% for diagnostic odds ratios according to the paper's thresholds. But some did change conclusions, and the principal error types—wrong numbers, incorrect arithmetic or derived values, mismatched groups, and misread reference standards—are exactly the sort of problems that a fluent summary can conceal.

What this changes: A review's evidence tier is not a guarantee that the data were faithfully lifted from a trial. The underlying source tables, participant counts and comparator direction have to survive an independent check.