Scientists should use AI as a tool, not an oracle

The article by Narayanan and Kapoor warns against over-reliance on AI in scientific research due to issues with hype and flawed methodology leading to non-reproducible results. They discuss the prevalence of data leakage and other errors, highlighting the need for better practices and scrutiny in using AI as a tool, rather than an oracle.

• AI model 'discovered' 2.2 million new materials was debunked.
• Leakage affects hundreds of papers across 30 disciplines.
• Reproducibility issues stem from misapplications of ML.
• 650 papers potentially underestimated the risk of leakage.
• REFORMS checklist developed to mitigate ML pitfalls.