concept · intermediate · partly written
Sakana's AI Scientist-v2 and its experiment tree
The AI Scientist-v2 stores each experiment as a node in a tree and had one of three generated papers accepted at a workshop; an outside test of version 1 found frequent coding failures.
Also called: AI Scientist, Sakana AI Scientist
How it holds knowledge
Each node holds a script, a plan, results and a vision-language-model review. Nodes flagged 'buggy' get debugging children, so a negative result is just a node that is not the best. 1
Reported, not yet replicated
What was shown, and what an outsider found more
One of three generated papers scored 6, 7 and 6 at an ICLR workshop and was withdrawn by pre-agreement; Sakana says none met its main-track bar, and a citation error was admitted. An independent evaluation of version 1 (abstract read) found 42% of experiments failed from coding errors; the version tested is not the v2 described here. 23
Reported, not yet replicated
A baseline that was reported more
Sakana reported trivial baselines for its automated paper reviewer (random and always-reject). Two readings of that figure disagree on the always-reject value (0.50 or 0.59), so neither is used here. 1
Unverified: from our own report, source not opened
Plain answer
The AI Scientist-v2 keeps an experiment tree of code, plan, results and review per node; it is a record of runs, not a store of claims. 1
Reported, not yet replicated
Why it matters
It is the one system here with an outside test that went badly, which shows what builders' own demos leave out. 3
Our inference
Still open here
- The Nature version dated 26 March 2026 was not opened: does it change any of this?
Not yet written: Equation, Real numbers.
Linked from
Sources
- Sakana AI and collaborators. The AI Scientist-v2: workshop-level automated scientific discovery via agentic tree search. arXiv 2504.08066 (2025). preprint, not peer reviewed; opened 2026-11-18 all records citing this
- The AI Scientist generates its first peer-reviewed workshop paper. Sakana AI (2025). organisation-page, not peer reviewed; opened 2026-11-18 all records citing this
- Beel J, Kan MY, Baumgart M. Independent evaluation of Sakana's AI Scientist (version 1). arXiv 2502.14297 (2025). preprint, not peer reviewed; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (6)
This record has been changed 6 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Status
- partly written
- Last updated
- 2026-11-19
- Record
- content/concepts/ai-scientist-v2-experiment-tree.json