Artificial intelligence has proven adept at diagnosing diseases, predicting protein structures, and even writing computer code. But a major new study reveals a critical limitation: AI cannot yet reliably predict which scientific studies will hold up when other researchers try to replicate them.

The findings, reported by the New York Times, represent a significant setback for efforts to address science's reproducibility crisis — the troubling pattern in which many published studies fail to produce the same results when repeated by independent teams. Over the past decade, high-profile replication failures have shaken fields from psychology to cancer biology, prompting calls for better methods to identify questionable research before it influences policy or clinical practice.

Researchers had hoped that machine learning algorithms, trained on features like sample sizes, statistical methods, and publication patterns, might flag studies likely to fail replication. Such a tool could theoretically help journals, funders, and scientists themselves prioritize which findings warrant the investment of follow-up research.

Instead, the study demonstrates that the factors determining whether an experiment will replicate remain too complex and context-dependent for current AI systems to parse. Subtle variations in experimental protocols, unmeasured confounding variables, and the irreducible role of chance all contribute to replication outcomes in ways that resist algorithmic prediction.

The research underscores a humbling reality: both generating reliable scientific knowledge and verifying it remain inherently difficult tasks that still require extensive human expertise and judgment. While AI continues to accelerate certain aspects of research — from literature review to data analysis — the core challenge of distinguishing robust findings from statistical flukes appears to resist automation.

For now, the scientific community will need to continue relying on traditional safeguards: pre-registration of study protocols, transparent data sharing, larger sample sizes, and the painstaking work of replication itself. The reproducibility crisis, it seems, won't be solved by an algorithm.