question · intermediate · partly written
Can an agent fetch the variant answer key from a public database during a run?
No benchmark opened blocks lookup of public variant databases, and no measurement of agents doing it was found.
What is known and what is not?
One benchmark gives no tools and one does not state its network access in the text read. Whether agents fetch public answer keys has not been measured in sources opened. 12
Reported, not yet replicated
Observation
What is known
What is not
What would settle it
Run agents with the answer-key domains reachable and blocked and compare scores and logs. 2
Our inference
Still open here
- Do agents with network access fetch public variant answer keys, and does blocking them change scores?
Not yet written: Hypotheses, What is missing, Who works on it.
Sources
- Queen, Zhang, Zou. CGBench: a benchmark of clinical genetics curation (exact title not recorded when read). arXiv 2510.11985 (2025). preprint, not peer reviewed; opened 2026-11-17 all records citing this
- VariantBench (Latch). Latch (2026). vendor-page, not peer reviewed; opened 2026-11-17 all records citing this
Where it sits on the map
History of this record (1)
This record has been changed 1 time, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.
- Kind
- big-open
- Register
- open-question
- Status
- partly written
- Last updated
- 2026-11-17
- Record
- content/questions/can-an-agent-fetch-the-variant-answer-key-during-a-run.json