Biology Atlas

Map / Open register

question · intermediate · partly written

Who validates language-model-proposed ontology terms when less confident raters accept fluent definitions?

Less confident raters scored AI-written definitions as good as human ones, which leaves open who checks them.

What is measured more

In one evaluation, less confident raters scored language-model definitions as good as human definitions; GPT-4 reached SubClassOf precision 0.894 and recall 0.5. 1

Reported, not yet replicated

What is missing

A validation protocol

None found in the source read. 1

Reported, not yet replicated

What is known

Raters of lower confidence are not a reliable check. 1

Reported, not yet replicated

What is not

How to validate at scale. 1

Reported, not yet replicated

Still open here

Not yet written: Observation, Hypotheses, What would settle it, Who works on it.

Linked from

Sources

  1. Toro S, Anagnostopoulos AV, Bello SM, Blumberg K, et al.. Dynamic Retrieval Augmented Generation of Ontologies using Artificial Intelligence (DRAGON-AI). Journal of biomedical semantics (2024). paper; opened 2026-11-18 all records citing this
Where it sits on the map
History of this record (5)

This record has been changed 5 times, on the dates below. The note kept against each change is written for the people who maintain the atlas, in their own shorthand, so it is not shown here; it is in the record itself.

Kind
big-open
Register
open-question
Triage
needs-research
Status
partly written
Last updated
2026-11-19
Record
content/questions/who-validates-language-model-proposed-ontology-terms.json