Over the past year, building AI Scientist agents with large language models (LLMs) has become a shared focus of academia and industry. From literature surveys and hypothesis generation to autonomously running experiments, and even discovering new algorithms and mathematical proofs that surpass decades of human effort, a growing body of evidence points to the potential of LLMs for automating scientific discovery.
But the core of scientific discovery has never been "fitting statistical patterns in data". It is about identifying the critical variables and uncovering the causal mechanisms behind them. An AI Scientist that cannot distinguish correlation from causation is easily misled by observational signals in the presence of hidden confounders, selection bias, and measurement noise, and may even reach erroneous conclusions with serious consequences. In a medical setting, for example, mistaking correlation for causation can directly lead to a wrong treatment plan.
This raises an immediate question: do the LLM agents currently used to build AI Scientists actually possess the ability to think causally?
To answer this question, researchers from MBZUAI, Carnegie Mellon University, the TMLR Group at Hong Kong Baptist University, the University of Oxford, and New York University propose CausalGame: a benchmark that evaluates the causal thinking capabilities of LLM agents through interactive games. The paper has been accepted at ICML 2026 and selected for an Oral presentation (roughly the top 0.7%).
Paper title: CausalGame: Benchmarking Causal Thinking of LLM Agents in Games
Authors: Zhenhao Chen*, Yongqiang Chen*, Chenxi Liu*, Junchi Yu, Xiangchen Song, Zijian Li, Jialin Li, Philip Torr, Bo Han, Kun Zhang (* denotes equal contribution and core contributors)
Project website: https://causalgame.github.io
Over the past two years, most work on AI Scientists has focused on different stages of the research pipeline: ideation, data analysis, code implementation, interactive discovery, and experiment design. These capabilities all matter, but they are not enough. The hardest part of scientific discovery is often not finding a correlation in the data, but realizing, precisely when a correlation looks perfectly plausible, that it may be an illusion manufactured by selection bias, measurement error, or hidden confounders. For an AI Scientist, this is exactly the threshold between "automating the research pipeline" and "genuine scientific discovery": can it tell correlation apart from causation?
For an AI Scientist that cannot make this distinction, the better it gets at automatically running experiments, the more systematically it may amplify wrong conclusions. If we decompose the scientific discovery capability of an AI Scientist into three levels:
Most existing benchmarks stay at the first two levels. CausalGame targets the third, the level at which AI Scientists remain furthest from genuine scientific discovery. As the paper emphasizes, scientific discovery ultimately seeks causal and mechanistic knowledge, that is, how a system would change under interventions and why, rather than correlations that hold only under a fixed data-generating process [1,2]. This positioning clearly separates it from prior work.
According to the comparison in Table 1 of the paper, CausalGame is the only benchmark that simultaneously covers all six dimensions: automated evaluation, experiment design, multi-turn interaction, causal relations, explanation evaluation, and observational pitfalls. It targets a more fundamental question: