concept · intermediate · partly written
MPRAbox and CodonBox: the two worked sandboxes in the Rao and Lander paper
MPRAbox scores designed DNA sequences with a damp oracle, and CodonBox hides an invented genetic code, so the two show how a score can rise without understanding.
How does MPRAbox score an agent?
The agent writes 50,000 DNA sequences of 200 bases. An oracle model labels them. A fresh network is trained on those labels and tested on 14 hidden sets. The agent sees 14 Pearson correlations and their mean. 12
Reported, not yet replicated
Is a 0.011 gap a result? more
The agent's median was 0.774 over five runs and the best human strategy's mean was 0.763 over five replicates. The text read gives no confidence interval for the difference. The repository ships per-seed result files, so anyone can compute the spread. The figures and supplementary figures of the paper were not viewed. 13
Reported, not yet replicated
Why does CodonBox saturate? more
A repeated character gives a repeated codon, and an all-hydrophobic chain reaches the maximum of 9 contacts. All eight invented worlds were run once each for 500 rounds. The hardest world was not solved. The authors read the notebooks because the score alone carries little signal. 12
Reported, not yet replicated
Plain answer
MPRAbox asks an agent to design DNA sequences that a regulatory-activity oracle scores. CodonBox asks it to find the rule of an invented genetic code. In MPRAbox, a language-model agent matched the best human strategy; in CodonBox, every run reached the top score within ten rounds without the rule being found. 1
Reported, not yet replicated
Why it matters
Real numbers
MPRAbox one-shot result
Claude Opus 4.7 scored a median Pearson r of 0.774 over five runs. Gemini 3.5 Flash scored 0.680 and GPT-5.5 scored 0.655. The best of 14 human strategies scored a mean of 0.763. 1
Reported, not yet replicated
CodonBox ceiling
Every run reached the maximum score of 9 within 10 rounds. 1
Reported, not yet replicated
Still open here
- Does the 0.011 gap survive a confidence interval computed from the per-seed files in the repository?
Not yet written: Equation.
Linked from
Sources
- Rao AS, Lander ES, Sabeti PC, et al.. Science sandboxes measure the scientific capability of AI agents. arXiv 2608.30165 (2026). preprint, not peer reviewed; opened 2026-11-17 all records citing this
- BroadBox: sandboxes. sandboxes.broadinstitute.org (2026). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
- science-sandbox (code repository for the sandboxes paper). GitHub (2026). web, not peer reviewed; opened 2026-11-17 all records citing this
- BroadBox hackathon. sandboxes.broadinstitute.org (2026). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)
This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Status
- partly written
- Last updated
- 2026-11-17
- Record
- content/concepts/broadbox-mprabox-and-codonbox.json