Biology Atlas

Map / Concepts

concept · intermediate · partly written

MPRAbox and CodonBox: the two worked sandboxes in the Rao and Lander paper

MPRAbox scores designed DNA sequences with a damp oracle, and CodonBox hides an invented genetic code, so the two show how a score can rise without understanding.

How does MPRAbox score an agent?

The agent writes 50,000 DNA sequences of 200 bases. An oracle model labels them. A fresh network is trained on those labels and tested on 14 hidden sets. The agent sees 14 Pearson correlations and their mean. 12

Reported, not yet replicated

Is a 0.011 gap a result? more

The agent's median was 0.774 over five runs and the best human strategy's mean was 0.763 over five replicates. The text read gives no confidence interval for the difference. The repository ships per-seed result files, so anyone can compute the spread. The figures and supplementary figures of the paper were not viewed. 13

Reported, not yet replicated

What labels do the hidden sets use? more

Five of the 14 sets use real assay labels and nine use labels from the oracle model, so most of the mean measures agreement with a model. The Table 1 caption says the genomic sets come from chromosomes excluded from the oracle's training. No analysis of whether designs resembling the oracle are favoured was found in the passages read. 1

Reported, not yet replicated

Why does CodonBox saturate? more

A repeated character gives a repeated codon, and an all-hydrophobic chain reaches the maximum of 9 contacts. All eight invented worlds were run once each for 500 rounds. The hardest world was not solved. The authors read the notebooks because the score alone carries little signal. 12

Reported, not yet replicated

What do the authors say limits the work? deep

They list the cost and noise of wet oracles, the inherited bias of damp ones, the thin complexity of dry ones, and the use of a single model in the long-horizon runs. 1

Reported, not yet replicated

Plain answer

MPRAbox asks an agent to design DNA sequences that a regulatory-activity oracle scores. CodonBox asks it to find the rule of an invented genetic code. In MPRAbox, a language-model agent matched the best human strategy; in CodonBox, every run reached the top score within ten rounds without the rule being found. 1

Reported, not yet replicated

Why it matters

These are the two worked examples in the paper the hackathon is built on. Their numbers show how large a gap the authors treat as a result. 14

Reported, not yet replicated

Real numbers

MPRAbox one-shot result

Claude Opus 4.7 scored a median Pearson r of 0.774 over five runs. Gemini 3.5 Flash scored 0.680 and GPT-5.5 scored 0.655. The best of 14 human strategies scored a mean of 0.763. 1

Reported, not yet replicated

CodonBox ceiling

Every run reached the maximum score of 9 within 10 rounds. 1

Reported, not yet replicated

Still open here

Not yet written: Equation.

Linked from

Sources

  1. Rao AS, Lander ES, Sabeti PC, et al.. Science sandboxes measure the scientific capability of AI agents. arXiv 2608.30165 (2026). preprint, not peer reviewed; opened 2026-11-17 all records citing this
  2. BroadBox: sandboxes. sandboxes.broadinstitute.org (2026). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
  3. science-sandbox (code repository for the sandboxes paper). GitHub (2026). web, not peer reviewed; opened 2026-11-17 all records citing this
  4. BroadBox hackathon. sandboxes.broadinstitute.org (2026). organisation-page, not peer reviewed; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)

This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Status
partly written
Last updated
2026-11-17
Record
content/concepts/broadbox-mprabox-and-codonbox.json