Biology Atlas

Map / Open register

question · intermediate · partly written

Can an agent fetch the variant answer key from a public database during a run?

No benchmark opened blocks lookup of public variant databases, and no measurement of agents doing it was found.

What is known and what is not?

One benchmark gives no tools and one does not state its network access in the text read. Whether agents fetch public answer keys has not been measured in sources opened. 12

Reported, not yet replicated

Observation

In a task where the agent has tools and the answer key is a public database page, the agent might read the key. No documented case was found. 12

Our inference

What is known

CGBench gives models the paper text and no tools. The text read of Latch's VariantBench does not say whether its sandbox had internet access. 12

Reported, not yet replicated

What is not

No benchmark opened blocks lookup of ClinVar, ClinGen or Gene2Phenotype pages during an agent run, and no run log was opened. 12

Our inference

What would settle it

Run agents with the answer-key domains reachable and blocked and compare scores and logs. 2

Our inference

Still open here

Not yet written: Hypotheses, What is missing, Who works on it.

Sources

  1. Queen, Zhang, Zou. CGBench: a benchmark of clinical genetics curation (exact title not recorded when read). arXiv 2510.11985 (2025). preprint, not peer reviewed; opened 2026-11-17 all records citing this
  2. VariantBench (Latch). Latch (2026). vendor-page, not peer reviewed; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)

This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Kind
big-open
Register
open-question
Status
partly written
Last updated
2026-11-17
Record
content/questions/can-an-agent-fetch-the-variant-answer-key-during-a-run.json